Image processing method and image processing device

ABSTRACT

An image processing method and device computes the image quality as an evaluation value, determine the image quality based on this evaluation value, and employ a restoration process in accordance with image quality when this image quality is poor. The image processing device includes an image quality computation unit to extract a characteristic amount to determine the image quality of the image data entered by an image input device and to compute the characteristic amount as the evaluation value, a binary threshold determination unit to determine a binary threshold value for the image to be processed using an evaluation value obtained by the image quality computation unit, and an image quality improvement unit to determine candidates for image quality improvement unit by extracting sections which have the possibility of image quality deterioration based on the characteristics and to perform image quality improvement on candidates for image quality improvement processing using the evaluation value obtained by said image quality computation unit. The image quality computation unit extracts a characteristic amount of the image in sections with the possibility of image quality deterioration unit determines the image quality using this characteristic amount as an evaluation value; the binary threshold value determination means determines the optimum binary threshold value on the basis of this evaluation value; and the image quality improvement unit extracts candidates for and performs the image quality improvement process only when image quality improvement is necessary.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a division of application Ser. No. 08/480,869, filedJun. 7, 1995, now U.S. Pat. No. 5,764,813. This application is alsorelated to application Ser. No. 08/478,784, filed Jun. 7, 1995 andtitled "Image Processing Method and Image Processing Device."

FIELD OF THE INVENTION

The present invention relates to an image processing method and an imageprocessing device which evaluates objectively the copy quality of a copymachine, for example, enables restoration of a deteriorated image, andfurther may be used as a preparatory process for an optical characterrecognition (OCR) process.

DESCRIPTION OF RELATED ART

The use of copy machines has spread widely in recent times. Thecapability of recent copy machines to restore images faithfully isimproving rapidly. Digital copy machines, in particular, which arebecoming more common today, can produce high quality copies with highlyfaithful copying capabilities.

Therefore, if an original to be copied has a clear and clean image,digital copy machines, with their highly faithful copying capabilities,can produce high quality copies that are virtually the same as theoriginal. However, when the original is one that was copied by aconventional analog copy machine and the image has deteriorated, adigital copy machine, with its highly faithful copying capabilities,will reproduce a deteriorated copy.

Moreover, in recent years, optical character recognition technology isspreading in which a printed text is read by a scanner and charactersfrom the input image data are extracted and changed into computer code.In such optical character recognition technology, a high ratio ofrecognition is achieved when a text is read by a scanner if the text isan original having a clear and clean character image as described inFIG. 41(a) rather than a copy produced by a copy machine.

However, even if the original has a clear and clean image, the imagedeteriorates with repeated copying and, if such a deteriorated image isread by a scanner, the copy may result in deteriorated images asdescribed in FIG. 41(b) and FIG. 41(c). FIG. 41(b) represents an imageproduced by reading a once-copied original by a scanner, while (c)represents an image produced by reading the copy of a once-copiedoriginal (twice-copied original) by the scanner.

In order to cope with the deterioration of image quality as a result ofcopying, various measures have been provided for an optical characterrecognition mechanism. A dictionary for character recognition, includingdeteriorated character images, may be prepared and/or a function torepair some degree of unevenness provided.

As described above, a high level of capability to reproduce a faithfulcopy is required of copy machines, but when a deteriorated image (FIG.41(b) and FIG. 41(c)) as a result of repeated copying is copied by adigital copy machine with a high level of capability to produce afaithful copy, there have been problems of inability to produce a highquality copy.

Moreover, evaluation of copy machine capability (evaluation of imagequality) has been primarily performed by subjective evaluation. In otherwords, in general, human eyes, after seeing the copied image, determinethe capability of the copy machine. Traditionally, there have beentechnologies associated with copy machines to correct the unevenness ofdeteriorated characters to some degree when characters which havedeteriorated as a result of repeated copying are used as the object ofoptical character recognition or further copying, but there has been notechnology to enable objective, numerical evaluation of image quality,to determine the quality of image, especially the so-called faintness orsmudge of the characters, and to repair appropriately according to thedeterioration condition of the image.

Moreover, in optical character recognition technology to extract andchange into codes the characters from input image data, deterioratedcharacters are not sufficiently recognized. For example, suppose thecharacter images of FIGS. 41(a), 41(b), and 41(c) were targeted forcharacter recognition. Assuming the character recognition rate of FIG.41(a) to be 100%, the rate drops rapidly with 90% for 41(b) and 80% for41(c). Thus, a sufficient recognition rate is not obtained fordeteriorated images. Moreover, images pleasing to the eye are notproduced.

Furthermore, in assigning binary values to character images, it isnecessary to determine the optimum binary threshold values, but thetraditional binary threshold determination method does not result inoptimum binary threshold values for characters. There have been cases inwhich faintness and smudges resulted after binary values were assignedto character images by final binary threshold values.

SUMMARY OF THE INVENTION

In order to solve these and other problems, the present inventionprovides an image processing method and an image processing devicewhich, applied to a copy machine, evaluates the quality of the copiedimage objectively, determines deteriorated sections of the image,enables execution of an improvement process for the deterioratedsections according to the deterioration condition, enables determinationof optimum binary threshold values for characters, and may be usedeffectively as a preparatory process for an optical characterrecognition device and the like.

An image processing method of the present invention comprises an imagequality computation process wherein a characteristic amount is extractedto determine the image quality of image data entered by an image inputdevice, and wherein the characteristic amount is computed as anevaluation value, and the image quality is determined by the evaluationvalue obtained by the image quality computation process.

Moreover, said computation process may include a first characteristicamount extraction process which has, to begin with, several patterns ofpixel characteristics as characteristic points; computes the firstcharacteristic amount, which is the ratio of the frequency of theappearance of said characteristic points in processing lines and thefrequency of reversal of black pixels and white pixels; and determinesimage quality using the first evaluation value which is the firstcharacteristic amount computed above.

Furthermore, said image quality computation process may include a secondcharacteristic amount extraction process which computes the averagelength of a continuous string of black pixels nearly equivalent to thesize of a character; computes the number of continuous strings of blackpixels longer than the average continuous string of black pixels nearlyequivalent to the size of a character; computes a second characteristicamount which is the ratio of the number of continuous strings of blackpixels longer than the average length and one-half of the number ofreversals of black pixel and white pixels; and determines image qualityusing a second evaluation value which is the second characteristicamount computed above.

In addition, said image quality computation process may include thefirst characteristic amount extraction process and said secondcharacteristic amount extraction process; compute evaluation valuesbased on a first evaluation value obtained by the first characteristicamount extraction process and a second evaluation value obtained by thesecond characteristic amount extraction process; and the image qualitymay be determined using an evaluation value based on the first and thesecond evaluation values.

Moreover, said image quality computation process may include a thirdcharacteristic amount extraction process to extract as the thirdcharacteristic amount the average length of a continuous string of blackpixels nearly equivalent to a character in the processing line and insaid first characteristic amount extraction amount; obtain an evaluationvalue based on a first evaluation value obtained from the firstcharacteristic amount extraction process and a third characteristicamount obtained from the third characteristic amount extraction process;and the image quality may be determined using an evaluation value basedon the first evaluation value and the third evaluation value.

Furthermore, said image quality computation process may include thesecond characteristic amount extraction process and the thirdcharacteristic amount extraction process; compute an evaluation valuebased on a second evaluation value obtained by the second characteristicamount extraction process and a third characteristic amount obtained bythe third characteristic amount extraction process; and the imagequality may be determined using an evaluation value based on the firstand the second evaluation values as well as the third characteristicamount.

In addition, said image quality computation process may include thefirst characteristic amount extraction process, the secondcharacteristic amount extraction process, and the third characteristicamount extraction process; compute evaluation values based on a firstevaluation value obtained by the first characteristic amount extractionprocess, a second evaluation value obtained by second characteristicamount extraction process, and a third characteristic amount obtained bythe third characteristic amount extraction process; and the imagequality may be determined using evaluation values based on the first andthe second evaluation values as well as the third characteristic amount.

Moreover, the image quality computation process may include a fourthcharacteristic amount extraction process which performs orthogonaltransformation of input image data into frequency regions in order toenable extraction of characteristic amount in the frequency space,computes a fourth characteristic amount by focusing on the highfrequency component after orthogonal transformation, and determines theimage quality using a fourth evaluation value which is the fourthcharacteristic amount computed above.

Furthermore, the image quality computation process may include a fifthcharacteristic amount extraction process which performs orthogonaltransformation of input image data into frequency regions in order toenable extraction of characteristic amount in the frequency space,computes a fifth characteristic amount by focusing on the low frequencycomponent after orthogonal transformation, and determines the imagequality using a fifth evaluation value which is the fifth characteristicamount computed above.

In addition, if the image quality computation process includes thefourth characteristic amount extraction process and the fifthcharacteristic amount extraction process, evaluation values may becomputed based on a fourth evaluation value obtained by the fourthcharacteristic amount extraction process and a fifth evaluation valuemay be obtained by the fifth characteristic amount extraction process,and the image quality may be determined using evaluation values based onthe fourth and the fifth evaluation values.

Moreover, the image quality computation process may include the fourthcharacteristic amount extraction process and the third characteristicamount extraction process, and compute evaluation values based on afourth evaluation value obtained by the fourth characteristic amountextraction process and a third characteristic amount obtained by thethird characteristic amount extraction process, enabling also todetermine the image quality using evaluation values based on the fourthevaluation value and the third characteristic amount.

Furthermore, the image quality computation process may include the fifthcharacteristic amount extraction process and the third characteristicamount extraction process, and compute evaluation values based on afifth evaluation value obtained by the fifth characteristic amountextraction process and a third characteristic amount obtained by thethird characteristic amount extraction process, enabling also todetermine the image quality using evaluation values based on the fifthevaluation value and the third characteristic amount.

In addition, the image quality computation process may include thefourth characteristic amount extraction process, the fifthcharacteristic amount extraction process, and the third characteristicamount extraction process; compute evaluation values based on a fourthevaluation value obtained by the fourth characteristic amount extractionprocess, a fifth evaluation value obtained by the fifth characteristicamount extraction process, and a third characteristic amount obtained bythe third characteristic amount extraction process; and the imagequality may be determined using evaluation values based on the fourthand fifth evaluation values as well as the third characteristic amount.

Moreover, the image quality computation process may impose restrictionson the range of extracting said characteristic amount if differentregions exist in an original to be processed and compute evaluationvalues by performing extraction of a characteristic amount for eachregion.

Furthermore, the image processing method of the present invention mayinclude an image quality computation process to extract a characteristicamount to determine the image quality of image data entered by an imageinput device and to compute an evaluation value which is the extractedcharacteristic amount, and an image quality improvement process todetermine, from the deterioration characteristic, candidates for theimage quality improvement process by extracting sections which have thepossibility of deteriorated image quality, and to execute the imagequality improvement process on candidates for image quality improvementprocessing by using evaluation values obtained by the image qualitycomputation process.

The image quality improvement process may include a processing candidateextraction process to extract candidates for image quality improvementprocessing, and pixel processing to interpolate pixels in executingimage quality improvement on processing candidates extracted by theprocessing candidate extraction process.

The processing candidate extraction process may include a characteristicpoint extraction process to detect and extract characteristic pointsgenerated by deterioration in a section with deteriorated image quality,and a candidate determination process to determine candidates for imagequality improvement using the positional relationship of thecharacteristic points extracted by the characteristic point extractionprocess.

Moreover, the pixel processing may include a threshold computationprocess which, using the evaluation value obtained by said image qualitycomputation process, obtains a threshold value from a function with theevaluation value as variable, compares the threshold value with theinterval on which interpolation of pixels is performed, and determineswhether or not to perform the interpolation process of pixels based onthe results of the comparison.

Furthermore, the pixel processing may include a character cutting-outprocess, also enabling execution of pixel interpolation within theregion of characters which are cut out by the character cutting-outprocess to improve image quality.

In addition, the image processing method of the present invention mayinclude an image quality computation process to extract a characteristicamount to determine the image quality of the image data entered by animage input device and to compute the characteristic amount as anevaluation value, and a binary threshold determination process todetermine the binary threshold value for the image to be processed usingthe evaluation value obtained by the image quality computation process.

The binary threshold value determination process may define a thresholdvalue which determines an evaluation value, based on more than one valueamong all the evaluation values obtained by said image qualitycomputation process, corresponding to a predetermined value as thetarget binary threshold value.

Moreover, the image processing method of the present invention mayinclude an image quality computation process to extract a characteristicamount to determine the image quality of the image data entered by animage input device and to compute the characteristic amount as anevaluation value, a binary threshold determination process to determinea binary threshold value for an image to be processed using anevaluation value obtained by the image quality computation process, andan image quality improvement process to determine candidates for theimage quality improvement process by extracting sections which have thepossibility of image quality deterioration based on the characteristicsand to perform the image quality improvement process on candidates forimage quality improvement processing using the evaluation value obtainedby said image quality computation process.

Furthermore, the image processing device of the present invention mayinclude an image input device to enter images written on originals andthe like, and an image quality computation unit to extract and compute acharacteristic amount as an evaluation value to determine the imagequality of the image data entered by said image input device.

The computation unit may include a first characteristic amountextraction device which has, to begin with, several patterns of pixelcharacteristics as characteristic points; computes the firstcharacteristic amount, which is the ratio of the frequency of theappearance of said characteristic points in processing lines and thefrequency of reversal of black pixels and white pixels; and determinesimage quality using the first evaluation value which is the firstcharacteristic amount computed above.

Furthermore, the image quality computation unit may include a secondcharacteristic amount extraction device which computes the averagelength of a continuous string of black pixels nearly equivalent to thesize of a character; computes the number of continuous strings of blackpixels longer than the average continuous string of black pixels nearlyequivalent to the size of a character; computes a second characteristicamount which is the ratio of the number of continuous strings of blackpixels longer than the average length and one-half of the number ofreversals of black pixel and white pixels; and determines image qualityusing a second evaluation value which is the second characteristicamount computed above.

In addition, the image quality computation unit may include the firstcharacteristic amount extraction device and the second characteristicamount extraction device; compute evaluation values based on a firstevaluation value obtained by the first characteristic amount extractiondevice and a second evaluation value obtained by the secondcharacteristic amount extraction device; and determine the image qualityusing an evaluation value based on the first and the second evaluationvalues.

Moreover, the image quality computation unit may include a thirdcharacteristic amount extraction device to extract as the thirdcharacteristic amount the average length of a continuous string of blackpixels nearly equivalent to a character in the processing line and inthe first characteristic amount extraction amount; obtain an evaluationvalue based on a first evaluation value obtained from the firstcharacteristic amount extraction device and a third characteristicamount obtained from the third characteristic amount extraction device;and the image quality may be determined using an evaluation value basedon the first evaluation value and the third evaluation value.

Furthermore, the image quality computation unit may include the secondcharacteristic amount extraction device and the third characteristicamount extraction device; compute an evaluation value based on a secondevaluation value obtained by the second characteristic amount extractiondevice and a third characteristic amount obtained by the thirdcharacteristic amount extraction device; and the image quality may bedetermined using an evaluation value based on the first and the secondevaluation values as well as the third characteristic amount.

In addition, the image quality computation unit may include the firstcharacteristic amount extraction device, the second characteristicamount extraction device, and the third characteristic amount extractiondevice; compute evaluation values based on a first evaluation valueobtained by the first characteristic amount extraction device, a secondevaluation value obtained by second characteristic amount extractiondevice, and a third characteristic amount obtained by the thirdcharacteristic amount extraction device; and the image quality may bedetermined using evaluation values based on the first and the secondevaluation values as well as the third characteristic amount.

Moreover, said image quality computation unit may include a fourthcharacteristic amount extraction device which performs orthogonaltransformation of input image data into frequency regions in order toenable extraction of characteristic amount in the frequency space,computes a fourth characteristic amount by focusing on the highfrequency component after orthogonal transformation, and determines theimage quality using a fourth evaluation value which is the fourthcharacteristic amount computed above.

Furthermore, the image quality computation unit may include a fifthcharacteristic amount extraction device which performs orthogonaltransformation of input image data into frequency regions in order toenable extraction of characteristic amount in the frequency space,computes a fifth characteristic amount by focusing on the low frequencycomponent after orthogonal transformation, and determines the imagequality using a fifth evaluation value which is the fifth characteristicamount computed above.

In addition, the image quality computation unit may include the fourthcharacteristic amount extraction device and said fifth characteristicamount extraction device and evaluation values may be computed based ona fourth evaluation value obtained by the fourth characteristic amountextraction device and a fifth evaluation value may be obtained by thefifth characteristic amount extraction device, and the image quality maybe determined using evaluation values based on the fourth and the fifthevaluation values.

Moreover, the image quality computation unit may include the fourthcharacteristic amount extraction device and the third characteristicamount extraction device, compute evaluation values based on a fourthevaluation value obtained by the fourth characteristic amount extractiondevice and a third characteristic amount obtained by the thirdcharacteristic amount extraction device, enabling also to determine theimage quality using evaluation values based on the fourth evaluationvalue and the third characteristic amount.

Furthermore, the image quality computation unit may include the fifthcharacteristic amount extraction device and the third characteristicamount extraction device, and compute evaluation values based on a fifthevaluation value obtained by the fifth characteristic amount extractiondevice and a third characteristic amount obtained by the thirdcharacteristic amount extraction device, enabling also to determine theimage quality using evaluation values based on the fifth evaluationvalue and the third characteristic amount.

In addition, the image quality computation unit may include the fourthcharacteristic amount extraction device, the fifth characteristic amountextraction device, and the third characteristic amount extractiondevice; compute evaluation values based on a fourth evaluation valueobtained by the fourth characteristic amount extraction device, a fifthevaluation value obtained by the fifth characteristic amount extractiondevice, and a third characteristic amount obtained by the thirdcharacteristic amount extraction device; and the image quality may bedetermined using evaluation values based on the fourth and fifthevaluation values as well as the third characteristic amount.

Moreover, the image quality computation unit may impose restrictions onthe range of extracting said characteristic amount if different regionsexist in an original to be processed and compute evaluation values byperforming extraction of a characteristic amount for each range.

Furthermore, the image processing method of the present invention mayinclude an image input device to enter image written on originals andthe like and an image quality computation unit to extract acharacteristic amount to determine the image quality of image dataentered by an image input device and to compute an evaluation valuewhich is the extracted characteristic amount, and an image qualityimprovement unit to determine, from the deterioration characteristic,candidates for the image quality improvement processing by extractingsections which have the possibility of deteriorated image quality, andto execute the image quality improvement processing on candidates forimage quality improvement processing by using evaluation values obtainedby said image quality computation unit.

The image quality improvement unit may include a processing candidateextraction device to extract candidates for image quality improvementprocessing, and pixel processing means to interpolate pixels inexecuting image quality improvement on processing candidates extractedby the processing candidate extraction device.

The processing candidate extraction device may include a characteristicpoint extraction device to detect and extract characteristic pointsgenerated by deterioration in a section with deteriorated image quality,and a candidate determination device to determine candidates for imagequality improvement using the positional relationship of thecharacteristic points extracted by the characteristic point extractiondevice.

Moreover, the pixel processing may include a threshold computation meanswhich, using the evaluation value obtained by said image qualitycomputation unit, obtains a threshold value from a function with theevaluation value as variable, compares said threshold value with theinterval on which interpolation of pixels is performed, and determineswhether or not to perform the interpolation process of pixels based onthe results of the comparison.

Furthermore, pixel processing means may include a character cut-outdevice, also enabling execution of pixel interpolation within the regionof characters which are cut out by the character cut-out device toimprove image quality.

In addition, the image processing device of the present invention mayinclude an image quality computation unit to extract a characteristicamount to determine the image quality of the image data entered by animage input device and to compute the characteristic amount as anevaluation value, and a binary threshold determination means todetermine the binary threshold value for the image to be processed usingthe evaluation value obtained by the image quality computation unit.

The binary threshold value determination unit may define a thresholdvalue which makes the evaluation value, based on more than one valueamong all the evaluation values obtained by said image qualitycomputation unit, correspond to the predetermined value as the targetbinary threshold value.

Moreover, the image processing device of the present invention mayinclude an image input device to enter images written on originals andthe like, an image quality computation unit to extract a characteristicamount to determine the image quality of the image data entered by theimage input device and to compute the characteristic amount as theevaluation value, an image quality improvement unit to determinecandidates for image quality improvement unit by extracting sectionswhich have the possibility of image quality deterioration based on thecharacteristics and to perform image quality improvement on candidatesfor image quality improvement processing using the evaluation valueobtained by said image quality computation unit, and a binary thresholdvalue determination unit to determine a binary threshold value for theimage to be processed using an evaluation value obtained by the imagequality computation unit.

The present invention includes an image quality computation unit todetermine the image quality of image data entered by an image inputdevice and to compute the result of the determination as an evaluationvalue corresponding to the image quality. The image quality computationunit may include a first characteristic amount extraction device whichhas, to begin with, several patterns of pixel characteristics ascharacteristic points; computes a first characteristic amount which isthe ratio of the frequency of the appearance of said characteristicpoints in the processing lines and the frequency of the reversal ofblack pixels and white pixels; and determines the image quality using afirst evaluation value which is the first characteristic amount computedabove. Moreover, the image quality computation unit may include a secondcharacteristic amount extraction device which computes the averagelength of a continuous string of black pixels nearly equivalent to thesize of a character, computes the number of continuous strings of blackpixels longer than the average continuous string of black pixels nearlyequivalent to the size of a character, computes a second characteristicamount which is the ratio of the number of continuous strings of blackpixels longer than the average length and one-half of the number ofreversals of black pixels and white pixels, and determines the imagequality using a second evaluation value which is the secondcharacteristic amount computed above.

The image quality computation unit may compute as the first evaluationvalue said first characteristic amount obtained by the firstcharacteristic amount extraction device, and as the second evaluationvalue said second characteristic amount obtained by the secondcharacteristic amount extraction device; and obtain an evaluation value,based on the first and the second evaluation values, which may be usedfor determination of the image quality. Moreover, said image qualitycomputation unit may include a first characteristic amount extractiondevice and a third characteristic amount extraction device. The thirdcharacteristic amount extraction device may obtain as the thirdcharacteristic amount the average length of a continuous string of blackpixels nearly equivalent to a character in the process line, and obtainan evaluation value, based on the first evaluation value and the thirdcharacteristic amount, which may be used to determine the image quality.

Moreover, the image quality computation unit may use the secondcharacteristic amount extraction device and a third characteristicamount extraction device, and compute an evaluation value, based on thesecond evaluation value and the third characteristic amount, which isused to determine the image quality.

Furthermore, the image quality computation unit may use the firstcharacteristic amount extraction device, the second characteristicamount extraction device, and the third characteristic amount extractiondevice; and compute an evaluation value, based on the first evaluationvalue, the second evaluation value, and a third characteristic amountobtained by said third characteristic amount extraction device, which isused to determine the image quality.

Moreover, the image quality computation unit may include a fourthcharacteristic amount extraction device; perform orthogonaltransformation of the input image data into frequency regions in orderto enable extraction of characteristic amounts in frequency space;compute a fourth characteristic amount by focusing on the high frequencycomponent after the orthogonal transformation to determine the imagequality using a fourth evaluation value, which is the fourthcharacteristic amount, and a fifth characteristic amount extractiondevice which performs orthogonal transformation of the input image datainto frequency regions; computes a fifth characteristic amount byfocusing on the low frequency component after the orthogonaltransformation; determine the image quality using a fifth evaluationvalue which is the fifth characteristic amount computed above; andfurther make it possible to obtain an evaluation value based on thefourth evaluation value obtained by the fourth characteristic amountextraction device and the fifth evaluation value obtained by the fifthcharacteristic amount extraction device, and to determine the imagequality using the evaluation value based on the fourth and the fifthevaluation values.

More specifically, the first characteristic amount and the fourthcharacteristic amount indicate the faintness of the characters, whilethe second characteristic amount and the fifth characteristic amountindicate smudging of the characters. From each of the characteristicamounts representing faintness and smudging, evaluation valuesassociated with faintness and smudging are obtained and the imagequality is determined objectively and accurately using these evaluationvalues individually or jointly.

Moreover, the image quality improvement unit may detect and extract, bythe processing candidate extraction device, characteristic points in theimage deteriorated sections produced by deterioration, and determinescandidates for the image quality improvement process using a positionalrelationship among the characteristic points extracted by thecharacteristic point extraction device. Then, by said pixel processingwork means, threshold values are obtained using the evaluation valuesobtained by said image quality computation unit and functions with theseevaluation values as variables. Next, the threshold values are comparedwith intervals on which interpolation of pixels is to be performed anddetermination is made from the comparison results as to whether or notinterpolation of pixels is to be conducted.

By these means, deteriorated sections such as faintness and smudging ofcharacters may be determined accurately. For a section with missingpixels thus determined as a deteriorated section, the interval in whichpixels are missing is compared with a predetermined threshold value andinterpolation is performed after determining if such an interpolation ofpixels is necessary. Thus, a restoration process matching the imagequality is realized.

Moreover, a characteristic amount may be extracted and computed as anevaluation value after determining the image quality of the image dataread in by an image input device and the threshold value, correspondingto a section in which an evaluation value, based on the two evaluationvalues associated with said faintness and smudging obtained above andmatching a certain predetermined value, may be specified as the binarythreshold value to be obtained. Thus, the binary threshold valueenabling the best image quality may be determined as the binarythreshold value to be obtained, and the binary change best suited forthe character may be realized.

Furthermore, by combining the image quality computation process, whichextracts a characteristic amount by determining the image quality of theimage data read in by said image input device and computes thecharacteristic amount as the evaluation value; the binary thresholdvalue determination process, which determines the binary threshold valuefor the image to be processed using the evaluation value obtained by theimage quality computation process; and the image improvement process,which determines candidates for the image quality improvement process byextracting a section with possible deterioration of image quality basedon its characteristics and performs the image quality improvementprocess on the candidates for image quality improvement process usingthe evaluation value obtained by said image quality computation process;objective and accurate determination of the image quality as a result offaintness and smudging may be realized. Moreover, the binary thresholdvalue to be obtained may be determined by an evaluation value based ontwo evaluation values representing the faintness and the smudging, whichenables optimum binary processing. In addition, the interpolationprocess can be performed if interpolation of pixels is found necessarywhen images after binary processing produce sections with missing pixelsdue to faintness.

BRIEF DESCRIPTION OF THE DRAWINGS

The invention will be described in detail with regard to the followingdrawings in which like reference numerals designate like elements andwherein:

FIG. 1 is a block diagram of a first aspect of the present invention.

FIG. 2(a) and FIG. 2(b) are an image and an enlarged view of a portionof an image, respectively, of the Chinese character "kura" showing"faintness."

FIG. 3(a)-3(f) are drawings showing characteristic points of faintnessrelated to black and white pixels.

FIG. 4 is a flow chart of a first characteristic amount extractionprocess for use in the first aspect of the present invention.

FIG. 5 is a drawing showing binary threshold values of faintness orsmudging from 5 to 12 for the chinese character "kura".

FIG. 6(a) and FIG. 6(b) are graphs showing the relationship between thebinary threshold value, the character recognition error rate, and thefirst evaluation value based on faintness.

FIG. 7 is a flow chart showing a second characteristic extractionprocess for use in the present invention.

FIG. 8(a) and FIG. 8(b) are graphs showing the relationship between thebinary threshold value, the character recognition error rate, and thesecond evaluation value based on "smudging."

FIG. 9 is a flow chart showing a computation process of the evaluationvalue A based on the first evaluation value and the second evaluationvalue for use in the present invention.

FIG. 10(a) and FIG. 10(b) are graphs showing the relationship betweenthe binary threshold value, the character recognition error rate, andthe evaluation value A (first evaluation value-second evaluation value)based on "faintness."

FIG. 11 is a graph showing the relationship between the binary thresholdvalue, the character recognition error rate, and the evaluation value B(first evaluation value+second evaluation value) based on "faintness."

FIG. 12 is a flow chart showing a computation process of the evaluationvalue C based on the first evaluation value, the second evaluationvalue, and the third characteristic amount for use in the presentinvention.

FIG. 13 is a flow chart showing a computation process of the fourthevaluation value in the frequency region for use in the presentinvention.

FIG. 14 is a drawing showing the high frequency components and the lowfrequency components in a matrix after Hadamard's transformation.

FIG. 15(a) and FIG. 15(b) are graphs showing the relationship betweenthe sequence and the ratio of dispersion in a certain binary thresholdvalue.

FIG. 16 is a graph showing the relationship between the binary thresholdvalue and the fourth evaluation value obtained in the frequency region(part 1).

FIG. 17(a) and FIG. 17(b) are graphs showing a distribution in a certainsequence other than the DC component in Hadamard's transformation.

FIG. 18 is a graph showing the relationship between the binary thresholdvalue and the fourth evaluation value obtained in the frequency region(part 2).

FIG. 19 is a flow chart of a computation process of the fifth evaluationvalue in the frequency region for use in the first aspect of presentinvention.

FIG. 20 is a graph showing the relationship between the binary thresholdvalue and the fifth evaluation value obtained in the frequency region(part 1).

FIG. 21 is a graph showing the relationship between the binary thresholdvalue and the fifth evaluation value obtained in the frequency region(part 2).

FIG. 22 is a flow chart showing a computation process of the evaluationvalue D or the evaluation value E based on the fourth evaluation valueand the fifth evaluation value for use in the present invention.

FIG. 23(a)(1)-FIG. 23(a)(3) and FIG. 23(b)(1)-FIG. 23(b)(3) are graphsrelated to the determination of an evaluation value and a correctedevaluation value respectively.

FIG. 24 is a flow chart showing a computation process of the evaluationvalue F based on the fourth evaluation value, the fifth evaluation, andthe third characteristic amount for use in the present invention.

FIG. 25(a) is a drawing of an original document for processing withdifferent regions for imposition of a restriction on the characteristicamount extraction range, and FIG. 25(b) and FIG. 25(c) are graphs ofevaluation values related to the different regions based on differentrestrictions.

FIG. 26 is a drawing showing an example of the detection of differentregions in an original document.

FIG. 27 is a block diagram including a second aspect of the presentinvention with the first aspect of the present invention.

FIGS. 28(a)-28(c) are a drawing showing conditions for interpolationcandidates in an example where characteristic points are facing eachother.

FIG. 29(a) and FIG. 29(b) are views, including enlarged views of aportion, of the Chinese character "kura" related to showingdeterioration of the image.

FIG. 30 is a graph indicating whether an interval between the pixels andthe corresponding section is an empty space due to deterioration or isinitially empty.

FIG. 31 is a graph showing an example for aiding in obtaining athreshold value TH3 using a function having evaluation values asvariables.

FIG. 32 is a flow chart showing the overall processing with the firstand second aspects of the present invention.

FIG. 33 is a drawing showing an example of adjacent charactersillustrating the execution of the pixel interpolation process after theprocess of character cut-out.

FIG. 34(a)(I)-FIG. 34(a)(III) and FIG. 34(b)(I)-FIG. 34(b)(III) aredrawings showing examples of improved image quality according to thesecond aspect of present invention.

FIG. 35 is a block diagram including a third aspect of the presentinvention with the first aspect of the present invention.

FIG. 36 is a flow chart showing the overall processing with the firstand third aspects of the present invention.

FIG. 37 is a graph of a binary threshold value versus error rate andevaluation value E.

FIG. 38 is a flow chart of a binary threshold value determinationprocess.

FIG. 39 is a block diagram of an image processing device combining animage quality computation unit, a binary threshold determination unit,and an image quality improvement unit.

FIG. 40 is a drawing showing the detection of the base line when thepresent invention is applied to alpha numeric characters.

FIG. 41(a)-FIG. 41(c) are drawings showing an example of an image of theChinese character "kura" as read by a scanner.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

Hereinafter, embodiments of the present invention will be described. Inthese embodiments, the resolution of the image is 300 dots per inch(DPI) and characters written in black against a white background (whitemanuscript paper) are used as the images to be processed.

As a first aspect of the present invention, a character image qualitycomputation unit to compute the degree of deterioration of image qualityof a character copied by a copy machine or a character read by a scannerprior to character recognition, for example, is described.

In FIG. 1, the image processing device 10 in the first aspect mainlycomprises an image quality computation unit 11 (to be explained indetail later), a CPU 12 which controls the entire process, and the RAM13. Moreover, an image input device 14 and an image output device 15 areconnected to the image processing device 10 and the image qualitycomputation unit 11, the RAM 13, the image input device 14, and theimage output device 15 are connected to the CPU 12 through the bus 16.

Said image input device 14 can be an optical image input device (ascanner, for example) if optical image input is to be performed, or acommunication input device or a memory device to store image data.Moreover the image output device 15 outputs (display, hard copy, orcommunication outputs) the image data entered after specifiedprocessing.

Moreover, the RAM 13 comprises a buffer for line data to store the image(character) data to be processed in the first aspect and the work areaneeded to perform various processes.

A first characteristic amount extraction device 111, a secondcharacteristic amount extraction device 112, a third characteristicamount extraction device 113, and a fourth characteristic amountextraction device 114, and a fifth characteristic amount extractionmeans 115 constitute the main components of said image qualitycomputation unit 11.

The image quality computation unit 11, as described before, computes thequality of the characters. Characteristic amounts such as faintness,smudging, and character size are the major criteria for determining thequality of the characters.

Said first characteristic amount extraction device 111 and a fourthcharacteristic amount extraction device 114 determine the image qualityby extracting the characteristic amount of faintness. Moreover, saidsecond characteristic amount extraction device 112 and a fifthcharacteristic amount extraction device 115 determine the image qualityby extracting the characteristic amount of smudging. Furthermore, saidthird characteristic amount extraction device 113 extracts thecharacteristic amount of length in the horizontal direction which isequivalent to character size and, by adding this characteristic amountto the two characteristic amounts of said faintness and smudging, itperforms more effectively the image quality improvement process in theimage quality improvement unit, which will be explained later.

Hereinafter, the characteristic amount extraction processes of thefirst-fifth amount extraction devices 111-115 will be describedindividually, but each of the first, second, and third characteristicamount extraction devices 111, 112, and 113 will be describedimmediately below, while the fourth and the fifth characteristic amountextraction device 114 and 115 will be explained later.

To begin with, the first characteristic amount extraction device 111will be explained.

As mentioned before, the first characteristic amount extraction device111 extracts the characteristic amount of faintness. Here, explanationis given with reference to FIG. 2 and FIG. 3.

FIG. 2(a) is an image of the Chinese character kura and has faintnessabout the level of the image shown in FIG. 41(c). FIG. 2(b) is amagnification of a certain section (circled area) where faintness isproduced in FIG. 2(a). FIG. 2(b) indicates that several pixels (1, 2, or3 pixels) in the vertical direction are in a horizontally convex shape.A horizontally convex shape refers to a horizontal protrusion of 1, 2,or 3 vertically contiguous pixels, with no additional pixels immediatelyabove or below them.

For example, consider the region enclosed by the broken line in FIG.2(b). Let L1 in the region be a focus line. In the focus line L1,sections K1, K2, and K3 present faintness. Here, K2 shows the conditionin which an empty space equivalent to 1 pixel is produced due to thefaintness of 1 pixel, while K3 shows an empty space equivalent to 2pixels due to the faintness of 2 pixels. Moreover, the convex part P1equivalent to 2 vertical pixels exists in the section showing faintnessK1, while the convex part P2 equivalent to 2 pixels and the convex partP3 equivalent to 1 pixel are facing each other in the section showingfaintness K2. Furthermore, the convex part P4 equivalent to 1 pixel andthe convex part P5 equivalent to 2 pixels are facing each other in thesection showing faintness K3.

Now the first characteristic amount is computed using these convexparts. The formula for computing the amount is as follows.

    First characteristic amount=Frequency of characteristic point appearance/Frequency of black and white reversal of pixels (1)

The first characteristic amount computed by formula (1) becomes one ofthe evaluation values by which excellence or poorness of quality isdetermined (hereafter referred to as the first evaluation value).

The characteristic points in said formula (1) represent thecharacteristics of said convex parts and are defined here as describedin FIG. 3(a)-3(f). FIG. 3(a) and FIG. 3(b) describe 1 pixelcharacteristic points (corresponding to the convex part of 1 pixel) withFIG. 3(a) containing a section in which pixels do not exist in thevertical direction and to the right in the diagram (hereafter referredto as an empty section), and FIG. 3(b) containing an empty section inthe vertical direction and to the left in the diagram. Moreover, FIG.3(c) and FIG. 3(d) describe 2 pixel characteristic points (correspondingto the convex part of 2 pixels in the vertical direction) with FIG. 3(c)containing an empty section in the vertical direction and to the rightin the diagram, and FIG. 3(d) containing an empty section in thevertical direction and to the left in the diagram. Furthermore, FIG.3(e) and FIG. 3(f) describe 3 pixel characteristic points (correspondingto the convex part of 3 pixels in the vertical direction) with FIG. 3(e)containing an empty section in the vertical direction and to the rightin the diagram, and FIG. 3(f) containing an empty section in thevertical direction and to the left in the diagram. Therefore, in orderto examine up through 3 pixel characteristic points, it becomesnecessary to provide pixel data equivalent to 5 lines (the regionenclosed by the broken line in FIG. 2(b), for example). In this case,the second line from the top of 5 lines is designated as the focus lineL1.

The frequency of appearance of pixel characteristic points on the focusline L1 in FIG. 2(b) is counted as follows: once for the convex part P1(equivalent to FIG. 3(d)), once for the convex part P2 (equivalent toFIG. 3(c)), once for the convex part P3 (equivalent to FIG. 3(b)), oncefor the convex part P4 (equivalent to FIG. 3(a)), and once for theconvex part P5 (equivalent to FIG. 3(d)), for a total of fiveappearances. Moreover, the frequency of black and white reversal ofpixels to be used in formula (1) is counted as follows: once for theconvex part P1 in the faintness section K1, once for the convex part P2in the faintness section K2, once for the convex part P3 in thefaintness section K2, once for the convex part P4 in the faintnesssection K3, once for the convex part P5 in the faintness section K3, andfinally, once in the last section of the character on the line for atotal of six reversals. Thus, the first characteristic amount isexpressed as first characteristic amount=5/6. However, this figure isobtained within the range described in FIG. 2(b) and, in reality, thefocus line L1 scans horizontally each line of the original to beprocessed, and the frequency is counted for the entire character on eachline. Thus, the first characteristic amount computed may naturally bedifferent from the value above.

FIG. 4 is a flow chart describing the process of computing the firstcharacteristic amount mentioned above. Here, the first characteristicamount computation process is performed with a scanner which scans andreads each line of the original text according to instructions by theline counter. To begin with, in FIG. 4 a black and white reversalcounter to count the frequency of black and white reversals of pixelsand a characteristic point counter to count the frequency of appearanceof characteristic points are initialized (step S1). Next, line data fromthe line specified by the line counter as line 5 are prepared (step S2),the second line is designated as the focus line to be scanned, and thefrequency of black and white reversals as well as the frequency of theappearance of characteristic points are counted on the line (step S3).In order to avoid duplicated counting in this case, counting is doneonly when the second line from the top in FIG. 3 matches the focus line.

As counting of the frequency of black and white reversals and thefrequency of the appearance of characteristic points are completed forthe focus line, the first characteristic amount is computed usingformula (1) (step S4). Then, determination is made on whether or not thefirst characteristic amount computation process is completed for theentire page of the original to be processed (step S5). If the process isnot completed, the line counter is incremented (step 6) and the processfrom step S2 through step S5 will be executed.

In other words, the line data from the line specified by the linecounter next to line 5 are prepared (step S2), and the second line isdesignated as the focus line to be scanned. After executing processessimilar to the above, the frequency of black and white reversals countedthis time is added to the frequency of black and white reversals of theprevious scanning, and a characteristic amount is computed using formula(1). In this manner, the first characteristic amount is computed for theentire page of the original to be processed. The reason for adding thenew frequency to the previous frequency is to improve the reliability ofthe evaluation value by increasing the amount of data. The same can besaid for computation of each evaluation value below.

In the process described above, the first characteristic amount iscomputed each time the black and white reversal frequency countingprocess and characteristic point appearance frequency counting arecompleted for a certain focus line. This is done to execute concurrentlythe image quality determination process in the image quality computationunit 11 and the image quality improvement process in the image qualityimprovement unit, to be explained later, or the binary threshold valuedetermination process in the binary threshold value determination unit.However, if the above image quality computation process is performed forthe entire image to be processed first, and then the improvement processor the binary threshold value determination process is performed later,the first characteristic amount may be computed after completion ofblack and white reversal frequency counting and characteristic pointappearance frequency counting for the entire page of the original to beprocessed in the process flow chart of FIG. 4.

Next, justification of said first characteristic amount in certaincharacter image samples (a character image sample containing the Chinesecharacter "kura" here) will be explained.

In FIG. 5, the character "kura" out of character image samples isscanned and input with 16 gradations (gray scale), and the characterimage is displayed wherein certain gradations (5-12 here) are changed tobinary values as the binary threshold value. In order to distinguishthis threshold value from several other threshold values to be mentionedin a later explanation, hereinafter the binary threshold value will bedenoted by TH1. Evaluation of said first characteristic amount is doneusing the character image whose image quality is changed with the abovemethod. Also, the character recognition rate is used as an evaluationmethod here. Incidently, in the evaluation using the characterrecognition rate, the rate may change even for the same image, accordingto the character recognition method used, but tendencies remain thesame.

FIG. 6 displays a graph (dotted line) of the error rate (non-recognitionrate) for each pixel with a binary threshold value TH1 from 5 through 12described in FIG. 5, and a graph (solid line) of the first evaluationvalue (this evaluation value is exactly the first characteristic amount)for each pixel with a binary threshold value TH1 from 5 through 12. Asdescribed in FIG. 6(a) an image with a binary threshold value of 7presents the lowest error rate. Moreover, the larger the value of thefirst evaluation value, the greater the level of faintness. Therefore,binary threshold values of 5 and 6 show a higher error rate due todeterioration of the image quality caused by faintness. Moreover, thereason for a large error rate for binary threshold values TH1 greaterthan or equal to 7 appears to be deterioration of the image quality dueto smudging.

The evaluation here is an evaluation based on faintness. In other words,the desired conclusion is that the greater the level of faintness, thelarger the value of the first evaluation value becomes, and with thisthe error rate also becomes larger. As described in FIG. 6(a), thisrelationship is established when the binary threshold value TH1 is lessthan 7. However, once the binary threshold value TH1 becomes 7 orgreater, the error rate becomes larger as the first evaluation valuebecomes smaller. The relationship between the first evaluation value andthe error rate reverses at either side of the binary threshold value TH1of 7. This is because the error rate caused by smudging increases whenthe binary threshold value TH1 becomes 7 or greater, as explainedbefore. In this manner, when the binary threshold value TH1 is 7 orgreater, deterioration of the image quality due to smudging occurs and,as described before, the relationship between the first evaluation valueand the error rate reverses at either side of the binary threshold valueTH1 of 7. In other words, faintness and smudging are in an opposingrelationship with the image quality, due to smudging becoming largerwith decreases in the first evaluation value. Since the evaluation inquestion here is based on faintness, first evaluation values for binarythreshold values TH1 7 or greater are folded back with respect to thebroken line in FIG. 6(a) and FIG. 6(b) is obtained. When a correlationis run for the error rate (dotted line) and the first evaluation value(solid line) in FIG. 6(b), the correlation coefficient is found to be0.86. The correlation coefficient assumes values less than or equal to 1and greater than or equal to -1 with the coefficient 1 indicating thehighest positive correlation. A coefficient of 1 shows that the twovariables are identical. Therefore, the correlation coefficient of 0.86shows that the first evaluation value and the error rate obtained herehave a similar relationship and that the first evaluation value is avalid index for determining the image quality.

Evaluating the first characteristic amount from the human eyesensitivity point of view, the severer the faintness in FIG. 5, thelarger the first evaluation value in FIG. 6(a), which is consistent withhuman eye sensitivity and establishes the first evaluation value as avalid index for determining image quality.

Thus, the level of faintness of characters is determined by the firstevaluation value.

Next, the second characteristic amount extraction device will bedescribed. The second characteristic amount is smudging. Here, imagequality will be determined by extracting the characteristic amount ofsmudging. A detailed explanation follows.

In FIG. 5 described above, as the binary threshold value TH1 becomeslarger, the number of horizontal black runs (refers to a continuousstring of black pixels) increases, which indicates that deterioration ofthe image quality due to smudging occurs. Thus, the image quality iscomputed using the number of horizontal black runs nearly equalling thesize of a character. The desired second characteristic amount iscomputed by the following formula.

    Second characteristic amount=Number of black runs longer than the specified length/Total number of black runs                         (2)

The value obtained from formula (2) becomes an evaluation value todetermine the excellence or poorness of the image quality. The "Numberof black runs longer than the specified length" here refers to thenumber of horizontal black runs nearly equal to the size of a character,and the number of black runs equal to or longer than a threshold valueTH2, to be described below, is counted. Here, a black run refers to acontinuous string of black pixels, as mentioned before.

To begin with, an initial value is established for the threshold valueTH2. The threshold value TH2 is established every time the processingline of the original to be processed is scanned, and the initial valueof TH2, TH0, is set as TH0=24 pixels. The 24 pixels here are equivalentto about 70% of the size of an 8 point character (3 mm square) if theresolution is 300 DPI. The reason for setting the initial value TH0 tobe 24 pixels is that characters smaller than that are not usually used.As mentioned above, the threshold value TH2 is reset each time theprocessing line of the original to be processed is scanned. In otherwords, the value is set to correspond to the characters actually beingused. For example, when each line of an original with much text isscanned, the initial value THO to scan the first line is 24 and theaverage length of black runs longer than or equal to 24 pixels iscomputed for each character on the line and the average value thusobtained is multiplied by a certain value a (0.6-1.0). Then a newthreshold value (threshold value TH2) is obtained using the thresholdvalue previously obtained as the initial value. Thus, the thresholdvalue TH2 represents the length of black runs nearly equivalent to thehorizontal length of an average character of each character up to thatpoint in a certain line.

Here, the threshold value TH2 is expressed as: TH2=average length ofblack runs, up to that point in a certain line, longer than or equal tothe second threshold value:

    TH2×α                                          (3)

In the formula (3), a is set at 0.6-1.0, but 0.85 was found to be mostappropriate in the experiments. By changing the threshold value TH2 withthe scanning of each line in this manner, it is possible to successfullycope with changes in the size of the characters.

In addition, the "total number of black runs" in formula (2) is thetotal number of black runs when the image is scanned in a sidewaysdirection, and because this numerical value is one-half the "frequencyof black and white reversals of pixels" in formula (1), this value mayalso be found using the "frequency of black and white reversals ofpixels" found in step S3 of the process flow in FIG. 4 by taking onehalf of this number.

FIG. 7 shows a flow chart of the processes of the second characteristicamount computation as described above. In FIG. 7, first the black andwhite reversal counter which counts the frequency of black and whitereversals; the run length register which stores the sum of black runlengths at least as long as threshold value TH2 in a line in order tofind the average length of black runs at least as long as thresholdvalue TH2 for each line; the run number counter which counts how manyblack runs at least as long as threshold value TH2 have occurred in eachline; a threshold value register in which is stored a threshold value(initially 24 pixels as the initial value TH0) which is the base used tofind threshold value TH2 for each line; and a row counter whichindicates the row that is the target of processing are all initialized(step S11).

Next, line data from the row indicated by the row counter through line 5is prepared (step S12), scanning of the lines is conducted with thetarget line being the second of these lines, and the frequency of blackand white reversals is counted in addition to the number of black runsat least as long as threshold value TH2 being detected (step S13). Inother words, in this step S13, in addition to the counting of thefrequency of black and white reversals, the threshold value TH2 is foundfrom formula (3) above with the value (initially 24) stored in thethreshold value register as the base in order to detect the number ofblack runs at least as long as threshold value TH2. The number of blackruns at least as long as this threshold value TH2 is detected, and thenumber of these black runs is counted by the run number counter.Furthermore, the contents of the threshold value register are updated inorder to make the threshold value found through this the base value(step S14).

Next, the second characteristic amount is detected using formula 2 above(step S15) on the basis of the value detected in step S13. Then, thedetermination is made (step S16) as to whether or not the process ofcomputing the second characteristic amount has been completed for theentire page of the document that is the object of processing, and if theprocess has not been completed, the row counter is incremented (stepS17), and the processes from step S12 through step S16 are conductedagain.

That is, line data from the row indicated by the row counter throughline 5 is prepared (step S12), scanning of the lines is conducted withthe target line being the second of these lines, processes similar tothose described above are conducted, and the number of black runs atleast as long as a preset length found this time is added to the numberof black runs at least as long as the preset length found the previoustime. Additionally, the total number of black runs counted this time isadded to the total number of black runs found the previous time, and thesecond characteristic amount is computed using formula (2). In this way,the process of computing the second characteristic value is conductedfor the entire page of the document that is the object of processing.

In addition, in the process which extracts this second characteristicamount, five lines need not be prepared for the line buffer number, forone line is sufficient, and in this case, the target line matches theline indicated by the row counter.

In the above processes, each time the processes in step S13 arecompleted, computation of the second threshold value is performed inorder to conduct the image quality determination process in the imagequality computation unit 11 in parallel with the image qualityimprovement process in the below-described image quality improvementunit or the binary threshold value determination process in the binarythreshold value determination unit. However, when the above-describedimage quality computation process is first conducted on the entire imagethat is the object of processing, and following this the image qualityimprovement process or the binary threshold value determination processis conducted, the processes in steps S13 and S14 are first conductedover the entire image being processed in the flow of processes in FIG.7, and after these processes have been entirely completed, computationof the second characteristic values may also be conducted all at once.

Next, the justification of this second characteristic amount(s) in aparticular sample character image (herein, a sample character imagecontaining the Chinese character "kura") will be described.

In this case also, the evaluation of this second characteristicamount(s) will be conducted with reference to FIG. 5 which was used inthe description of the justification of the above-described firstcharacteristic amount. As this method of evaluation, here alsoevaluation will be conducted using the character recognition rate. Evenin evaluations using the character recognition rate, the recognitionrate will change even with the same image depending on the characterrecognition method, but the tendency will be the same in both.

FIG. 8(a) shows the relationship (indicated by the dotted line) of theerror rate (the rate of inability to recognize) with respect to eachimage with a binary threshold value TH1 of 5 to 12, as shown in FIG. 5,and the relationship (indicated by the solid line) of the evaluationvalue (here, evaluation value is the value of the second characteristicamount, and will hereinafter be called the second evaluation value) withrespect to each image with a binary threshold value TH1 of 5 to 12. Ascan be seen from this drawing, the error rate is lowest when the imagehas a binary threshold value TH1 of 7. In addition, the drawingindicates that smudging and faintness are larger the higher thenumerical value of the second evaluation value. Accordingly, in the caseshown in FIG. 8(a), the error rate is high due to deterioration of imagequality caused by faintness when the binary threshold value TH1 is 5 and6, and in addition, the error rate is high due to deterioration of imagequality caused by smudging when the binary threshold value TH1 is 7 orhigher.

Here, the evaluation is an evaluation which has "smudging" as the basis.That is to say, this is an evaluation wherein the desired evaluationvalue is one wherein the numerical value of the second evaluation valuebecomes larger the larger the "smudging", and the error rate also risesaccompanying this. As can be seen from FIG. 8(a), this relationship isestablished when the binary threshold value TH1 is larger than 7.However, the relationship is such that the error rate becomes larger asthe second evaluation value becomes smaller when the binary thresholdvalue TH1 is smaller than 7, so that the relationship between the errorrate and the second evaluation value is reversed, with the boundarybeing a binary threshold value TH1 of 7. This is because the error rateincreases due to "faintness," as has been explained above, when thebinary threshold value TH1 is smaller than 7.

In this way, when the binary threshold value TH1 is smaller than 7,deterioration of image quality is produced due to faintness the smallerthe second evaluation value becomes, and the relationship between theerror rate and the evaluation value is reversed, with the boundary beinga binary threshold value TH1 of 7 as has been explained above. Here,this evaluation is an evaluation which has "smudging" as the base, andin the case of "smudging," because the relationship is such that theerror rate increases with increases in the numerical value of the secondevaluation value, when the second evaluation value when the binarythreshold value TH1 is smaller than 7 is folded back with linearsymmetry about the dashed line in FIG. 8(a), FIG. 8(b) results. Fromthis FIG. 8(b), when the correlation between the error rate (indicatedby the dotted line) and the second evaluation value (indicated by thesolid line) is found, the correlation coefficient becomes 0.70. Inparticular, the correlation coefficient between the two is 0.98 when thebinary threshold value TH1 is in the smudging region greater than 7, theerror rate indicated by the dotted line and the second evaluation valueindicated by the solid line have substantially same relationship, andthis becomes particularly effective in detecting "smudging."

In addition, if this evaluation of the second characteristic amount isevaluated from sensitivity as seen by the naked eye, the larger smudgingbecomes in FIG. 5, the larger the second evaluation value becomes inFIG. 8(a), and this matches the sensitivity seen by the naked eye, whichindicates that this second evaluation value is an adequate indicator indetermining image quality.

In this way, it is possible to determine the degree of smudging of acharacter from the second evaluation value.

Furthermore, an evaluation value is computed which takes intoconsideration both the first characteristic amount and the secondcharacteristic amount found from the above-described processes. That isto say, by computing a new evaluation value which takes intoconsideration "faintness," which is the first characteristic amount, and"smudging," which is the second characteristic amount, a more accurateimage quality determination is conducted. The following formula is anexample of the method of computing this new evaluation value. Here, thenew evaluation value is evaluation value A.

    (Evaluation value A)=(First characteristic amount)-(second characteristic amount)                                                   (4)

FIG. 9 shows a flow chart of the processes of computing this evaluationvalue A. In FIG. 9, first each of the registers and counters indicatedin the flowcharts in FIG. 4 and FIG. 7 above are initialized (step S21).Next, the line data is prepared through line five from the row indicatedby the row counter (step S22), scanning of the line(s) is conducted withthe second line being the target line, and the above-described firstcharacteristic amount and second characteristic amount are computed(step S23).

Furthermore, each time these first characteristic amount and secondcharacteristic amount are computed, the evaluation value A is computedusing above-described formula (4) (step S24). Next, the determination ismade (step S25) as to whether the process of computing evaluation valueA has been completed for the entire page of the document that is theobject of processing, and if this process has not been completed, therow counter is incremented (step S26), and the processes inabove-described steps S22 through S25 are conducted again.

In the above processes, each time the processes in step S23 arecompleted, the evaluation value A is computed in order that this kind ofimage quality determination process in this image quality computationunit 11 is conducted in parallel with the below-described image qualityimprovement processes in the image quality improvement unit or thebinary threshold value determination processes in the binary thresholdvalue determination unit. However, when the image quality computationprocess is first conducted as described for the entire page of thedocument that is the object of processing, following which all of theimage quality improvement processes or the binary threshold valuedetermination processes are conducted, in the process flow of FIG. 9 allevaluation values A may be computed after the processes in step S23 havebeen completed for the entire image that is the object of processing.

Next, the justification for this evaluation value A in a particularsample character image (herein, a sample character image containing theChinese character "kura") will be described, said value taking intoconsideration these first and second characteristic amounts.

In this case also, the evaluation of this evaluation value A will beconducted with reference to FIG. 5 which was used in the description ofthe justification of the above-described first and second characteristicamounts. As this method of evaluation, here also evaluation will beconducted using the character recognition rate. Even in evaluationsusing the character recognition rate, the recognition rate will changeeven with the same image depending on the character recognition method,but the tendency will be the same in both.

FIG. 10(a) shows the relationship (indicated by the dotted line) of theerror rate (the rate of inability to recognize) with respect to eachimage with binary threshold values TH1 of 5 to 12, as shown in FIG. 5,and the relationship (indicated by the solid line) of the evaluationvalue A with respect to each image with binary threshold values TH1 of 5to 12. As can be seen from FIG. 10(a), the error rate is lowest when theimage has a binary threshold value TH1 of 7. In addition, because inthis case the evaluation value A is value found by subtracting thesecond characteristic amount from the first characteristic amount, forexample the first characteristic amount (first evaluation value) is 0.11from FIG. 6(a) when the binary threshold value TH1 is 5, and the secondcharacteristic amount (second evaluation value) at this time is 0 fromFIG. 8(a). Accordingly, the evaluation value A, taking intoconsideration the first and second characteristic amounts, is 0.11.

In addition, the first characteristic amount (first evaluation value) is0.04 from FIG. 6(a) when the binary threshold value TH1 is 7, and thesecond characteristic amount (second evaluation value) at this time is0.02 from FIG. 8(a). Accordingly, the evaluation value A is 0.02. Inaddition, the first characteristic amount (first evaluation value) is0.02 from FIG. 6(a) when the binary threshold value TH1 is 12, and thesecond characteristic amount (second evaluation value) at this time is0.09 from FIG. 8(a). Accordingly, the evaluation value A is -0.07. Inthis way, the evaluation value A is computed.

In FIG. 10(a), the error rate is high due to deterioration of imagequality caused by faintness when the binary threshold value TH1 is 5 and6, and in addition, the error rate is high due to deterioration of imagequality caused by smudging when the binary threshold value TH1 is 7 orhigher.

In the case shown in this drawing also, the relationship between theerror rate and the second evaluation value is reversed, with theboundary being a binary threshold value TH1 of 7. Here, because thesecond characteristic amount relating to "smudging" is subtracted fromthe first characteristic amount relating to "faintness", it is possibleto think of this as having "faintness" as a standard in this case also,so similar to above-described FIG. 6(b), FIG. 10(b) results when theevaluation value at or above a binary threshold value of 7 is foldedback around the dashed line in FIG. 10(a). From this FIG. 10(b), whenthe correlation between the error rate (indicated by the dotted line)and the evaluation value A (indicated by the solid line) is found, thecorrelation coefficient becomes 0.90. In this way, by determining imagequality through detecting the evaluation value A, which takes intoconsideration the first and second characteristic amounts, a highercorrelation coefficient is obtained than when the first characteristicamount or the second characteristic amount is used independently,indicating that this becomes an extremely reliable indicator as adetermination of image quality.

When computing the evaluation value, which takes into consideration"faintness", which is the first characteristic amount, and "smudging",which is the second characteristic amount, in the above-describedexample a new evaluation value was computed by finding the difference ofthe first characteristic amount and the second characteristic amount,but a new evaluation value (called evaluation value B) may also be foundfrom the sum of the first characteristic amount and the secondcharacteristic amount using the formula below.

    (Evaluation value B)=(First characteristic amount)+(second characteristic amount)                                                   (5)

The flowchart of the processes of computing this evaluation value B isbasically the same as FIG. 9, but the process in step S24 differs inthat each time the first characteristic amount and the secondcharacteristic amount are computed, evaluation value B is computed fromformula (5) in place of above-described formula (4).

In addition, in the above processes, each time the processes in step S23are completed, evaluation value B is computed in order that this kind ofimage quality determination process in this image quality computationunit 11 is conducted in parallel with the below-described image qualityimprovement processes in the image quality improvement unit or theoptimum binary threshold value determination processes in the binarythreshold value determination unit. However, when the image qualitycomputation process is first conducted as described for the entire pageof the document that is the object of processing, following which all ofthe image quality improvement processes or the binary threshold valuedetermination processes are conducted, in the process flow of FIG. 9 allevaluation values B may be computed after the processes in step S23 havebeen completed for the entire image that is the object of processing.

Next, the justification of this evaluation value B in a particularsample character image (herein, a sample character image containing theChinese character "kura") will be described, said value taking intoconsideration these first and second characteristic amounts.

In this case also, the evaluation of this evaluation value B will beconducted with reference to FIG. 5. As this method of evaluation, herealso evaluation will be conducted using character recognition rate. Evenin evaluations using the character recognition rate, the recognitionratio will change even with the same image depending on the characterrecognition method, but the tendency will be the same in both.

FIG. 11 shows the relationship (indicated by the dotted line) of theerror rate (the rate of inability to recognize) with respect to eachimage with binary threshold values TH1 of 5 to 12, as shown in FIG. 5,and the relationship (indicated by the solid line) of the evaluationvalue B to each image with binary threshold values TH1 of 5 to 12. Ascan be seen from FIG. 11, the error rate is lowest when the binarythreshold value TH1 is 7. In addition, because in this case theevaluation value B is a value found by adding the second characteristicamount to the first characteristic amount, for example the firstcharacteristic amount (first evaluation value) is 0.11 from FIG. 6(a)when the binary threshold value TH1 is 5, and the second characteristicamount (second evaluation value) at this time is 0 from FIG. 8(a).Accordingly, the evaluation value B, taking into consideration the firstand second characteristic amounts, is 0.11. In addition, the firstcharacteristic amount (first evaluation value) is 0.04 from FIG. 6(a)when the binary threshold value TH1 is 7, and the second characteristicamount (second evaluation value) at this time is 0.02 from FIG. 8(a).Accordingly, the evaluation value B is 0.06. In addition, the firstcharacteristic amount (first evaluation value) is 0.02 from FIG. 6(a)when the binary threshold value TH1 is 12, and the second characteristicamount (second evaluation value) at this time is 0.09 from FIG. 8(a).Accordingly, the evaluation value B is 0.11. In this way, the evaluationvalue B is computed.

From this FIG. 11, when the correlation between the error rate(indicated by the dotted line) and the evaluation value B (indicated bythe solid line) is found, the correlation coefficient is high at 0.96.In this way, by determining image quality through detecting theevaluation value B, which takes into consideration the first and secondcharacteristic amounts, a higher correlation coefficient is obtainedthan when the first characteristic amount or the second characteristicamount is used independently, indicating that this becomes an extremelyreliable indicator as a determination of image quality.

In addition, as is clear from FIG. 11, the binary threshold value whereevaluation value B is smallest is 7, and because the best image qualitycan be obtained when the binary threshold value is 7, the binarythreshold value corresponding to the point where the evaluation value Bis smallest may be chosen as the threshold value for binary coding. Inother words, when the first characteristic amount and the secondcharacteristic amount are added together, the point where the evaluationvalue B thus obtained is smallest is the threshold value with the bestimage quality, and because of this it is possible to determine easilythe threshold value for binary coding by selecting said threshold valueas the threshold value for binary coding (a description of thisthreshold value for binary coding is provided hereinafter). For example,in the case of FIG. 10(a), when the threshold value is larger than theevaluation value which is a certain standard, faintness is produced,while when the threshold value is smaller than an evaluation value whichis a certain standard, smudging is produced, but because it is difficultto specify where to choose the evaluation value which is that certainstandard, promptly obtaining the optimum threshold value for binarycoding is not possible with the degree of ease in FIG. 11. Similarly,with FIG. 6(a) and FIG. 8(a) also, because it is difficult to specifywhere to choose the evaluation value which is a certain standard,promptly obtaining the optimum threshold value for binary coding is notpossible with the degree of ease indicated in FIG. 11.

Next, an evaluation value which makes the processes in thebelow-described image quality improvement unit more effective isobtained by adding one more characteristic amount (called the thirdcharacteristic amount) to the above-described first characteristicamount and second characteristic amount. This third characteristicamount is computed by the third characteristic amount extraction device113 in FIG. 1, and this characteristic amount is the length in thesideways direction corresponding to the size of the character.

With the below-described image quality improvement unit, processes whichrestore missing pixels are primarily conducted, but in this case, thenumber of missing pixels differs depending on the character size. Inother words, large characters have a large number of missing pixels,while small characters have only a small number of missing pixels. Thus,by making a new evaluation value C out of the value found using thecharacter size, which is this third characteristic amount, in additionto the above-described first and second characteristic amounts andevaluation values A and B, it becomes possible to conduct a moreeffective process in the image quality improvement unit.

This third characteristic amount is an average length of black runs atleast as long as a fixed length, and as explained above, this is a valueclose to the size of the character. The average length of black runs atleast as long as a fixed length is the length of the average of "blackruns at least as long as a fixed length" used when finding theabove-described second characteristic amount. In addition, an evaluationvalue C is computed using the following formula.

    (evaluation value C)=(first and second evaluation values or evaluation values A and B)×(third characteristic amount)       (6)

FIG. 12 shows a flowchart of the process of computing this evaluationvalue C. In FIG. 12, first each of the registers and counters areinitialized (step S31), similar to the processes shown in the flowchartin above-described FIG. 9. Next, the line data is prepared through linefive from the row indicated by the row counter (step S32), scanning ofthe line(s) is conducted with the second line being the target line, andthe above-described first characteristic amount, second characteristicamount and third characteristic amount are computed (step S33).

Furthermore, each time these first, second and third characteristicamounts are computed, the evaluation value C is computed usingabove-described formula (6) (step S34). Next, the determination is made(step S35) as to whether or not the process of computing evaluationvalue C has been completed for the entire page of the document that isthe object of processing, and if this process has not been completed,the row counter is incremented (step S36), and the processes inabove-described steps S32 through S35 are conducted again.

In the above processes, each time the processes in step S33 arecompleted, the evaluation value C is computed in order that this kind ofimage quality determination process in this image quality computationunit 11 is conducted in parallel with the below-described image qualityimprovement processes in the image quality improvement unit or theoptimum binary threshold value determination processes in the binarythreshold value determination unit. However, when the image qualitycomputation process is first conducted as described for the entire pageof the document that is the object of processing, following which all ofthe image quality improvement processes or the binary threshold valuedetermination processes are conducted, in the process flow of FIG. 11all evaluation values C may be computed after the processes in step S33have been completed for the entire image that is the object ofprocessing.

Now, the following four patterns can be considered for this evaluationvalue C.

(a) (evaluation value C)=(first characteristic amount (first evaluationvalue))×(third characteristic amount)

(b) (evaluation value C)=(second characteristic amount (secondevaluation value))×(third characteristic amount)

(c) (evaluation value C)=(evaluation value (evaluation value A) takingthe second characteristic amount into consideration in addition to thefirst characteristic amount)×(third characteristic amount)

(d) (evaluation value C)=(evaluation value (evaluation value B) takingthe second characteristic amount into consideration in addition to thefirst characteristic amount)×(third characteristic amount)

The above (a) is called evaluation value C1, (b) is called evaluationvalue C2, (c) is called evaluation value C3 and (d) is called evaluationvalue C4.

In this way, by using the third characteristic amount, which ischaracter size, it is possible to more efficiently conduct image qualityimprovement processes in the below-described image quality improvementunit.

Extraction of the various characteristic amounts as described above wasconducted in actual space (the image as seen by the naked eye), but nexta description will be given of extraction of characteristic amounts infrequency space.

First, a description will be given of a method of extracting acharacteristic amount (called the fourth characteristic amount) relatingto "faintness" in this frequency space. In extracting the characteristicamount relating to "faintness" in this frequency space, first adescription will be given of the relationship between the frequency andthe characteristic amount which is "faintness."

As can be seen from FIG. 41, when "faintness" is produced, the edgeportion of lines become unclear. The edges becoming unclear can bethought of as a decrease in the high frequency component. Accordingly,it is possible to know the degree of faintness by paying attention tothe high frequency component.

In light of this, a description is now provided of the method ofextracting the characteristic amount (fourth characteristic amount)relating to "faintness" in the frequency space. Extraction of thisfourth characteristic amount is conducted by the fourth characteristicamount extraction device 114 in FIG. 1. Fourier transformation, discretecosine transformation, and Hadamard's transformation are among themethods of orthogonal transformation to frequency space, and anyorthogonal transformation method may be used, but here, the descriptionwill be given for an example using a Hadamard's transformation with thecomputation processes limited to only addition and subtraction.

Roughly, in the order of processes in this Hadamard's transformation, asshown in the flowchart in FIG. 13, first the pixels that are the objectsof processing are divided into blocks of size n pixels by m pixels (stepS41), and a orthogonal transformation (a Hadamard's transformation inthis case) is conducted (step S42) with respect to the separated blocks.The size of this division is arbitrary, but here it is convenient if nand m are in the range 1-32. Furthermore, computation of the fourthcharacteristic amount is conducted (step S43), paying attention to thehigh frequency component (called high sequences in Hadamard'stransformation) obtained from the Hadamard's transformation. Adescription of the method of computing this fourth characteristic amountis given below.

This Hadamard's transformation is conducted as shown by the followingformula.

    Y=H.sub.n XH.sub.m                                         (7)

In the above formula, Y represents the matrix with n rows and m columnsfollowing a Hadamard's transformation, H_(n) and H_(m) represent thematrices of dimension n and m respectively used in the Hadamard'stransformation, and X represents the n pixel by m pixel sectional imagematrix. Here, the matrices H_(n) and H_(m) used in the Hadamard'stransformation when n and m are both 4 pixels (dimension 4) are:##EQU1##

In addition, X, which indicates the sectional image matrix, is an n×mmatrix represented by data indicating the black or white state of eachpixel (0 for white, 1 for black).

The n row by m column matrix Y found in this way following theHadamard's transformation (here, n and m are both 4 pixels) has theproperties that said matrix yields higher frequency components (highersequences) the larger the numerical values n and m become, as shown inFIG. 14, and in addition yields lower frequency components (lowsequences) the smaller the numerical values of n and m become.Accordingly, in the n row by m column matrix Y following the Hadamard'stransformation (here, n and m are both 4 pixels), it is desirable to payattention to the sections with large numerical values n and m whenlooking for high frequency components, and it is preferable to payattention to the sections with small numerical values n and m whenlooking for low frequency components. Specifically, in the case where nand m relates to 4 pixels (n=m=4), the low frequency components are whenn and m are 0 or 1 as indicated by the dotted line in FIG. 14, and thehigh frequency components are when n or m is 2 or 3.

With this Hadamard's transformation, the n row by m column matrix Yfollowing the Hadamard's transformation can be divided into blocks. Thatis to say, if N is the number of block divisions, the n row by m columnmatrices Y1, Y2, . . . , YN are obtained following the Hadamard'stransformation. Here, representing each entry in the matrices Y1, Y2, .. . , YN following the Hadamard's transformation as y_(ij) (0≦i≦n-1,0≦j≦m-1), the variance σ_(ij) in each entry is represented by thefollowing formula. ##EQU2##

In addition, calling the total variance σ_(x), the ratio ρ_(ij) of thevariance in each entry y_(ij) of the matrices Y1, Y2, . . . , YNfollowing the Hadamard's transformation can be defined as:

    ρ.sub.ij =σ.sub.ij /σ.sub.x                (10)

Here, the total variance σ_(x) of formula (10) is represented by:##EQU3##

FIG. 15(a) and FIG. 15(b) are graphs indicating the ratio of thevariance in each sequence (frequency component) when faintness isproduced by copying a particular sample image. Here, n=m=8, so thesequences in both the row direction and the column direction are from 0to 7, and in addition, taking the number of copies to be 0 (thiscorresponds to a binary threshold value TH1 of 7 in FIG. 5), taking thenumber of copies to be 1 (this corresponds to a binary threshold valueTH1 of 6 in FIG. 5), and taking the number of copies to be 2 (thiscorresponds to a binary threshold value TH1 of 5 in FIG. 5), in FIG.15(a) the relationship between the ratio of the variance and eachsequence when the binary threshold value TH1 is 7 is indicated by thesolid line, the relationship between the ratio of the variance and eachsequence when the binary threshold value TH1 is 6 is indicated by thedotted line, and the relationship between the ratio of the variance andeach sequence when the binary threshold value TH1 is 5 is indicated bythe dashed line. In addition, in FIG. 15(a) the data is data when adocument with horizontal writing is input in a vertical direction andscanned in the horizontal direction (the row direction). FIG. 15(a)represents the relationship between the variance ratio and each sequencefor the components (i=0, j=0 to 7) in the horizontal direction (rowdirection) while FIG. 15(b) represents the relationship between thevariance ratio and each sequence for the components (i=0 to 7, j=0) inthe vertical direction (column direction).

Here, because this is a case wherein a document written horizontally hasbeen inserted in the vertical direction and is scanned in the horizontaldirection (row direction), faintness is easily produced in thehorizontal direction (row direction), and consequently, the differencein number of copies appears as the difference in the degree of faintnessin the horizontal direction (row direction), as can be seen from FIG.15(a) and FIG. 15(b). That is, in FIG. 15(a), when considering the highsequence (sequence 4) sections, it can be seen that the difference inthe number of copies appears as the difference in the degree offaintness. Thus it is possible to find an evaluation value (thisevaluation value will be called the fourth evaluation value) relating to"faintness", paying attention to these high sequence sections.

The following methods are given as methods of computing this fourthevaluation value.

(I) The sum of the variance ratios in the high sequence sections becomesthe fourth evaluation value. That is to say, the fourth evaluation valueis expressed as: ##EQU4##

This takes into consideration all variance ratios in the high frequencycomponents shown in FIG. 14, but in cases wherein the direction offaintness is known beforehand, for example in the above-described casewherein faintness is easily produced in the horizontal direction, thefourth evaluation value may take into consideration only FIG. 15(a)(take into consideration only the j component), so that the value isfound from: ##EQU5##

Because this formula (13) uses only the numerical value (variance ratio)clearly expressing the difference in number of copies as the differencein the degree of faintness, it is possible to make the precision of theevaluation value that is found high. In formulas (12) and (13), areciprocal expression is used in order to make the evaluation value alarge numerical value in comparison with the size of the faintness. FIG.16 is used to describe the relationship between the above-describedbinary threshold value TH1 (TH1=5 to 12) and the fourth evaluation valuefound from formula (13). Here, because the block size is n×m=8×8, thehigh sequence sections are in the range n/2 to n-1 from formula (13),that is to say, sequences 4 to 7. Now, taking the variance ratios in thehorizontal direction with respect to sequences 4 to 7 at a binarythreshold value TH1 of 5 to be, for example, 0.0055 at sequence 4,0.0070 at sequence 5, 0.0068 at sequence 6, and 0.0039 at sequence 7 asfound through calculations, the sum of these becomes 0.0232. Thereciprocal of this is 43.1, and this value becomes the fourth evaluationvalue. Similarly, supposing that 0.0288 is found through calculations tobe the sum of the variance ratios in the horizontal direction withrespect to sequences 4 to 7 at a binary threshold value TH1 of 6, thereciprocal is 34.7 and this value becomes the fourth evaluation value.In addition, supposing that 0.0431 is found through calculations to bethe sum of the variance ratios in the horizontal direction with respectto sequences 4 to 7 at a binary threshold value TH1 of 7, the reciprocalis 23.2 and this value becomes the fourth evaluation value. The curve(FIG. 16) of these fourth evaluation values found in this manner atbinary threshold values TH1 of 5 to 7 traces a curve similar to thecurve of the first evaluation value at binary threshold values TH1 of 5to 7 shown in above-described FIG. 6(a).

From FIG. 15(a), only evaluation values in the range of binary thresholdvalues TH1 from 5 to 7 have been found, but it is clear that even atbinary threshold values TH1 of 8 and greater, a curve is traced which issimilar to that in above-described FIG. 6(a), and in fact it is possibleto obtain a curve similar to that in FIG. 6(a) in the range of binarythreshold values TH1 from 5 to 12. In addition, the curve indicated bythe dotted line in FIG. 16 shows the sum (see FIG. 15(b)) of varianceratios in the vertical direction with respect to sequences 4 to 7 atbinary threshold values TH1 of 5 to 12, and consequently, becausefaintness is slight in the vertical direction in this case, not a largedifference is created in the respective evaluation values in the rangeof binary threshold values from 5 to 12.

The fourth evaluation value can be found as described above, but thismethod uses many sum computations in order to find the variance ratio ascan be seen from formulas (12) and (13), and because of this thefollowing method is given as a simpler method.

(II) The ratio of sum of the absolute values of the high sequencesections in the matrix Y (matrices Y1 to YN) following the Hadamard'stransformation to the sum of the absolute values of all components inmatrix Y (matrices Y1 to YN) following the Hadamard's transformation isfound, and the fourth evaluation value is found on the basis of thisratio.

This method (II) is described below.

The above-described variance ratios being large means that the variancesare relatively large. In the case of a Hadamard's transformation, theaverage values can be approximated with a Laplace distribution about 0,as shown in FIG. 17, with the exception of the DC component (i=0, j=0).From this, when the variance ratio is relatively small (faintness islarge) at the high sequence sections in FIG. 15(a), such as in the caseof the image with a binary threshold value TH1 of 5, the width of thedistribution about 0 is relatively small as in FIG. 17(a), and as aresult, the sum of the absolute values of the high sequence sections inmatrix Y (matrices Y1 to YN) following the transformation also becomesrelatively small.

On the other hand, when the variance ratio is relatively large(faintness is small) at the high sequence sections, such as in the caseof the image with a binary threshold value TH1 of 7, the width of thedistribution about 0 is relatively large as in FIG. 17(b), and as aresult, the sum of the absolute values of the high sequence sections inmatrix Y (matrices Y1 to YN) following the transformation also becomesrelatively large.

FIG. 17(a) shows the distribution of the values of the high sequencesections (here, the values of each Y corresponding to sequence 4) of thematrix Y (matrices Y1 to YN) following the transformation at a binarythreshold value TH1 of 5 in FIG. 15(a), while FIG. 17(b) shows thedistribution of the values of the high sequence sections (here, thevalues of each Y corresponding to sequence 4) of the matrix Y (matricesY1 to YN) following the transformation at a binary threshold value TH1of 7 in FIG. 15(a). As can also be seen from FIG. 17(a) and FIG. 17(b),comparing images with large faintness and images with small faintness inthe high sequences (sequence 4 in this case), the images with largefaintness have a smaller width of distribution than do the images withsmall faintness, and as a result of this, the sum of the absolute valuesin all high sequence components is relatively small in the case of theimages with large faintness in contrast to the images with smallfaintness. From this, it is possible to compute an evaluation value onthe basis of the sum of the absolute values in all high sequencecomponents in the matrix Y (matrices Y1 to YN).

Accordingly it is possible to express the fourth evaluation value withthe following formula. ##EQU6##

In formula (14), Σ|y_(ij) | expresses the total sum of the ij componentsof the matrix Y (matrices Y1 to YN) following the transformation of eachblock in the N blocks. Furthermore, if the direction of faintness isknown beforehand, as noted above, for example when it is known thatfaintness is easily produced in the horizontal direction as noted above,it is fine for only the j component to be taken into consideration, inwhich case formula (14) is found from: ##EQU7##

Because this formula (15) uses only the numerical value (variance ratio)which clearly expresses the difference in the number of copies as adifference in the degree of faintness, it is possible to make theprecision greater for the evaluation value that is found. The reasonreciprocals are used in formulas (14) and (15) is so that the evaluationvalue is a large numerical value in comparison to the size of thefaintness.

FIG. 18 shows the relationship between the above-described binarythreshold value TH1 (TH1=5 to 12) and the fourth evaluation valueobtained from above-described formula (15). Here, the size of thedivision blocks is n×m=8×8, and in addition, the number N of blocks is3. When the binary threshold value TH1 is 6, the y_(0j) (0≦j≦7)components of matrix Y1 after the transformation in the first block are(14, 6, -5, 2, 1, -2, -2, 1), the y_(0j) (0≦j≦7) components of matrix Y2after the transformation in the second block are (13, -7, 0, -4, 0, 2,0, -1), and the y_(0j) (0≦j≦7) components of matrix Y3 after thetransformation in the third block are (12, 5, -3, 0, -2, 1, 0, 1).

Now, the sum of the absolute values of all components of matrix Y1 afterthe transformation in the first block is, from formula (15),14+6+5+2+1+2+2+1=33, and in addition, the sum of the absolute values ofthe high sequence sections (from sequence 4 up) out of these is1+2+2+1=6. In addition, the sum of the absolute values of all componentsof matrix Y2 after the transformation in the second block is 27 using asimilar calculation, and the sum of the absolute values of the highsequence components out of these is 3. Furthermore the sum of theabsolute values of all components of matrix Y3 after the transformationin the third block is 24, and the sum of the absolute values of the highsequence components out of these is 4.

Accordingly, the fourth evaluation value when the binary threshold valueTH1 is 6 is, from formula (15), fourth evaluation value=1/ (sum ofabsolute values of high sequence components)/(sum of absolute values ofall components)!, and when the above numerical values are substitutedinto this formula, the fourth evaluation value becomes 1/(6+3+4)/33+27+24)!, and thus it is found that the fourth evaluationvalue is about 6.6.

FIG. 18 is a graph showing the fourth evaluation values as found usingformula (15) for the cases when the binary threshold value TH1 is 5 andwhen the binary threshold value TH1 is 7, similar to the case explainedabove. The curve in this FIG. 18 traces a curve that is similar to thatof FIG. 6. In addition, the curve indicated by the dotted line in FIG.18 indicates the evaluation values with respect to the verticaldirection in each image when the binary threshold value TH1 is 5 to 12,and in this case, because the faintness in the vertical direction issmall, no large difference is produced in the evaluation values in eachof the images at binary threshold values of 5 to 12.

By thus extracting the characteristic amounts in the frequency space asdescribed above, it is possible to compute a fourth evaluation valuerelating to "faintness." However, because this description in FIGS. 16and 18 is of data for the case wherein a document written horizontallyis inserted in the vertical direction and scanned in the horizontaldirection (row direction) as explained above, the evaluation value ofeach image is the result of the effects of faintness in the horizontaldirection. Accordingly, if the results of FIG. 16 and FIG. 18 areconsidered, it is possible to determine the direction in which faintnessis produced, or the direction of insertion of the image (insertion inthe vertical direction or insertion in the horizontal direction).

In addition, with the above description, a description was given onlyfor the horizontal direction and the vertical direction, but it ispossible to detect the direction in which faintness is produced from thematrix Y following transformation, said direction being in the rangefrom 0° to 90°. For example, it is possible to investigate the 45°direction by finding the values from the i=j (0≦i≦n-1) components in thematrix Y following transformation.

In formula (12), a fourth evaluation value is computed taking intoconsideration the variance ratio of all elements in the high frequencycomponents, and in addition, in formula (13), a fourth evaluation valueis computed taking into consideration the variance ratio of all elementsin the high frequency components in the direction in which faintness isproduced, but the fourth evaluation value may also be computed takinginto consideration the variance ratio of only one or two or moreelements in the high frequency components. The same is also true offormulas (14) and (15).

The above description is a description for a method of extracting acharacteristic amount (the fourth characteristic amount) relating to"faintness" in a frequency space, but next, a method will be describedfor extracting a characteristic amount (called the fifth characteristicamount) relating to "smudging" in the frequency space. In extracting thecharacteristic amount relating to "smudging" in this frequency space,first a description will be given of the relationship between thefrequency and the characteristic amount which is "smudging."

As described above, the occurrence of smudging consists of pixelsbecoming black and the frequency of black and white reversals of thepixels becoming small, and greater smudging can be considered to occurwhen the frequency of black and white reversals becomes smaller.Thinking of this frequency of black and white reversals as thefrequency, the sections where smudging is produced can be considered thelow frequency components, and it is possible to know the degree ofsmudging by considering these low frequency components.

A description will be given of the method of extracting thecharacteristic amount (fifth characteristic amount) relating to"smudging" in the frequency space. Extraction of this fifthcharacteristic amount is conducted by the fifth characteristic amountextraction device 115 in FIG. 1. Fourier transformation, discrete cosinetransformation, and Hadamard's transformation are among the methods oforthogonal transformation to frequency space as noted above, and anyorthogonal transformation method may be used, but here, the descriptionwill be given for an example using a Hadamard's transformation with thecomputation processes limited to only addition and subtraction.

Basically, in the order of processes in this Hadamard's transformation,as shown in the flowchart in FIG. 19, first the pixels that are theobjects of processing are divided into blocks of size n pixels by mpixels (step S51), and a orthogonal transformation (a Hadamard'stransformation in this case) is conducted (step S52) with respect to theseparated blocks. The size of this division is arbitrary, but here it isconvenient if n and m are in the range 1-32. Furthermore, computation ofthe characteristic amount (called the fifth characteristic amount) isconducted (step S53), paying attention to the low, frequency components(low sequences) or the DC component (direct current component, that isto say, the i=j=0 component) of the matrix Y following thetransformation which is obtained from the Hadamard's transformation.

As the method of computing this fifth evaluation value, there are:

(I) Considering the DC component in the matrix Y (Y1-YN) following theHadamard's transformation, the fifth evaluation value is set, forexample, as the variance value (found using formula (9) above) in thisDC component.

The numerical value of this DC component is a value which corresponds tothe number of black pixels in each block. Hence, the distribution widthof the numerical value in this DC component becomes wider the moresmudging is produced, and as a result, the variance becomes larger.Accordingly, by paying attention to this DC component, it is possible toset an evaluation value relating to "smudging."

(II) The evaluation value is computed paying attention to the lowfrequency components (low sequences) in the matrix Y (matrices Y1 to YN)following the Hadamard's transformation.

This uses the variance value (found from formula (9) above) of the lowsequences (sequences 1 to 3 when the block size is n=m=8) other than theDC component as the evaluation value. Or, it is also possible to use thesum of the variance values of two or more elements of the low sequences(sequences 0 to 3) including the DC component as the evaluation value.

FIG. 20 shows the change in the variance value with respect to the imagecondition (binary threshold values TH1 of 5 to 12) in the low sequences(sequences 0 to 3) in the matrix Y following the Hadamard'stransformation (with the size of 1 block being n×m=8×8), and as is clearfrom this graph, the variance value is larger the larger the smudging ofthe image is. Accordingly, in this case the variance value in the DCcomponent may be set as the evaluation value considering only this DCcomponent as indicated by method (I) above, and in addition, it is alsopossible to use the variance value of any of the low sequences otherthan the DC component (sequences 1 to 3) as the evaluation value.Furthermore, it is also possible to use the sum of the variance valuesof two or more elements in the low sequences including the DC component(sequences 0 to 3) as the evaluation value.

(III) The evaluation value is set as the sum of the absolute values ofeach component of the low frequency components (low sequences) in thematrix Y (Y1 to YN) following the Hadamard's transformation.

As can be seen from above-described FIG. 20, the variance value becomeslarger the larger the smudging is. The variance value being large meansthat the width of the distribution of numerical values for each y_(ij)in the matrix Y (Y1 to YN) following the transformation is wide. Fromthis, the variance being large (large smudging) in the low sequencessuch as in the case of the image when the binary threshold value TH1 is12, for example, means that the width of the distribution of numericalvalues is large, and as a result, the sum of the absolute values of eachcomponent in the matrix Y (Y1 to YN) following transformation alsobecomes large. In contrast, the variance values being small in the lowsequences (small smudging) means that the width of the distribution ofnumerical values is small, and as a result the sum of the absolutevalues of each component in the matrix Y (Y1 to YN) followingtransformation also becomes small.

Hence, it is possible to extract a characteristic amount relating to"smudging" from the sum of the absolute values of each component in thematrix Y (Y1 to YN) following transformation in the low sequences, andit is also possible to compute the average of these values (the fifthevaluation value). The fifth evaluation value is found from thefollowing formula. ##EQU8##

In this formula (16), division by the number N of blocks is used inorder to prevent change in the size of the evaluation value because ofthe number of blocks (in order to normalize the numerical value). In theabove formula (16), it is also possible to use only values in thedirection of faintness in order to simplify the process. Here, "thedirection of faintness" is used regardless of the fact that this is aformula for computing an evaluation value relating to smudging in orderthat this coincide with the above-described process of computing"faintness", and in addition so that the determination as to whether ornot more smudging is being produced in the direction of ease of smudgingcan be conducted with ease. Accordingly, when faintness is noticeablyproduced in the horizontal direction similar to above, the above formula16 can be expressed as: ##EQU9##

FIG. 21 explains the relationship between the above-described binarythreshold value TH1 (TH1=5 to 12) and the fifth evaluation valueobtained from this formula (17). Here, the size of the division blocksis n×m=8×8, and in addition, the number N of blocks is 3. When thebinary threshold value TH1 is 6, the y_(0j) (0≦j≦7) components of matrixY1 following the transformation in the first block are (14, 6, -5, 2, 1,-2, -2, 1), the y_(0j) (0≦j≦7) components in matrix Y2 following thetransformation in the second block are (13, -7, 0,-4, 0, 2, 0, -1), andthe y_(0j) (0≦j≦7) components of matrix Y3 following the transformationin the third block are (12, 5, -3, 0, -2, 1, 0, 1).

From formula (17), the sum of the absolute values in the low sequences(sequences 0 to 3) of matrix Y1 following the transformation in thefirst block is 14+6+5+2=27. By a similar calculation, the sum of theabsolute values in the low sequences in matrix Y2 following thetransformation in the second block is 24. Furthermore, the sum of theabsolute values in the low sequences in matrix Y3 following thetransformation in the third block is similarly 20.

Hence, the evaluation value for the case where the binary thresholdvalue TH1 is 6 is, from formula (17), the fifth evaluation value= (sumof the absolute values of the low sequence components)/(number N ofblocks)!, and when the above-described numerical values are substitutedin, the fifth evaluation value becomes (27+24+20)/3, so that this fifthevaluation value is found to be about 23.7. Because there are inactuality a good number of white sections on the surface of the documenton which is drawn the image that is the object of processing, the actualnumerical value is a value smaller than the above numerical value, butbecause no problem at all arises in this description if the above notednumerical value is used as shown, the numerical value that is the resultof the above calculations is used as shown to facilitate thisdescription.

Similarly, FIG. 21 is a graph of the evaluation values found usingformula (17) for binary threshold values TH1 from 5 to 12. The curve inthis FIG. 21 traces out the same curve as in FIG. 8(a).

By thus extracting a characteristic amount in the frequency space asdescribed above, it is possible to compute a fifth evaluation valuerelating to "smudging."

In formula (16), a fifth evaluation value is computed taking intoconsideration all elements in the low frequency components, and inaddition, in formula (17), a fifth evaluation value is computed takinginto consideration all elements in the low frequency components in thedirection in which faintness is produced, but the fifth evaluation valuemay also be computed taking into consideration only one or two or moreelements in the low frequency components.

Next, an evaluation value will be computed which takes intoconsideration both the fourth characteristic amount and the fifthcharacteristic amount found in the processes above. In other words, amore accurate image quality determination will be conducted by computingan evaluation value which takes into consideration "faintness," which isthe fourth characteristic amount, and "smudging," which is the fifthcharacteristic amount. Among the methods of computing this evaluationvalue, there is a method wherein the difference of the fourth evaluationvalue and the fifth evaluation value is found and a method wherein thesum of the fourth evaluation value and the fifth evaluation value isfound, but here the description will be for the case wherein the sum ofthe fourth evaluation value and the fifth evaluation value is found. Theevaluation value found from the difference of the fourth evaluationvalue and the fifth evaluation value will be called evaluation value D,and the evaluation value found from the sum of the fourth evaluationvalue and the fifth evaluation value will be called evaluation value E.Evaluation value E is computed using the following formula.

    Evaluation value E=(Fourth characteristic amount (fourth evaluation value))+(fifth characteristic amount (fifth evaluation value)) (18)

FIG. 22 shows a flowchart of the process of computing this evaluationvalue E. In FIG. 22, first the pixels that are the objects of processingare divided into blocks of size n pixels by m pixels (step S61), and aorthogonal transformation (a Hadamard's transformation in this case) isconducted (step S62) with respect to the divided blocks. Furthermore,the evaluation value E is computed, taking into consideration theabove-described fourth evaluation value and fifth evaluation value (stepS63).

In conducting computations with the above formula (18), when theabsolute values of the two evaluation values differ by a large amount,the effects of the larger absolute value are felt while it is impossibleto effectively utilize the evaluation value with the smaller absolutevalue. For example, as shown in FIG. 23(a)(1)-FIG. 23(b)(3), FIG.23(a)(1) is a curve (corresponding to FIG. 18) showing faintness whileFIG. 23(a)(2) is a curve (corresponding to FIG. 21) showing smudging,and the absolute values of these evaluation values differ greatly. Whenthe two values are added with this kind of large difference between theabsolute values of the evaluation values, a curve such as the one shownin FIG. 23(a)(3) results. In other words, over the interval where thebinary threshold value TH1 is 5 to 7, faintness is large and theevaluation value should also be a large value, but an irregularityresults because the effects of the larger absolute value are felt whileit is impossible to effectively utilize the evaluation value with thesmaller absolute value.

Accordingly, in this kind of case, a correction of evaluation values isconducted so that the absolute values of each of the evaluation valuesare equal at some standard binary threshold value (here, TH1=7) and sothat the range of change of the two values is also the same. Thefollowing example uses the values (see FIG. 18) for the fourthevaluation value obtained from method (II) in the description of theextraction of the fourth characteristic amount, and the values (see FIG.19) for the fifth evaluation value obtained from method (III) in thedescription of the extraction of the fifth characteristic amount.

The above-described correction of the evaluation value is conducted asdescribed below. That is to say, the new evaluation value followingcorrection is found from:

    (new evaluation value)=({(Maximum of other evaluation value)-(minimum of other evaluation value)}/{(maximum of evaluation value being corrected)-(minimum of evaluation value being corrected)})×((evaluation value)-(standard value of evaluation value being corrected))+(standard value of other evaluation value) (19)

Here, "evaluation value being corrected" indicates the evaluation valuewhich is being corrected, while "other evaluation value" indicates theevaluation value not being corrected. In addition, standard value refersto the evaluation value at the binary threshold value TH1 which is thestandard. For example, in the case shown in FIG. 23(a)(1)-FIG. 23(a)(3),calling the evaluation value for "faintness" shown in FIG. 23(a)(1) the"evaluation value being corrected", and calling the binary thresholdvalue TH1 of 7 the standard, the result is:

    (new evaluation value)={(50-20)/(8-4)}×(evaluation value-5)+25

In this formula, when the evaluation value 5 at the binary thresholdvalue of 7 is substituted in as the evaluation value in (evaluationvalue-5), the new evaluation value that should be found at the binarythreshold value TH1 of 7 is found to be 25. In addition, when theevaluation value 8 at the binary threshold value TH1 of 5 is substitutedin as the evaluation value in (evaluation value-5), the new evaluationvalue that is to be found at the binary threshold value TH1 of 5 isfound to be 47.5, and in addition, when the evaluation value 4 at thebinary threshold value TH1 of 12 is substituted in as the evaluationvalue in (evaluation value-5), the new evaluation value that is to befound at the binary threshold value TH1 of 12 is found to be 17.5,yielding the curve for the new evaluation value after correction shownin FIG. 23(b)(1).

Accordingly, by adding the values in FIG. 23(b)(1) and FIG. 23(b)(2),the curve for evaluation value E for binary threshold values TH1 of 5 to12 shown in FIG. 23(b)(3) is obtained. By making this correction on astandard image, it becomes possible to use the other image as shown.

As can be seen from the curve for evaluation value E found in this wayby adding the fifth characteristic amount (fifth evaluation value) tothe fourth characteristic amount (fourth evaluation value), the binarythreshold value TH1 where the image quality is best is the location withthe smallest evaluation value, and from this fact it is possible todetermine the optimum binary threshold value TH1 upon binary coding bydetecting the place where the evaluation value is smallest. Hence, it ispossible to conduct determination of the binary threshold value at thetime of binary coding of characters with simplicity and moreover withprecision.

Next, an evaluation value is obtained which makes the process in thebelow-described image quality improvement unit more effective, saidvalue obtained by adding one more characteristic amount (theabove-described third characteristic amount) to the above-describedfourth characteristic amount and fifth characteristic amount. Asdescribed above, this third characteristic amount is computed by thethird characteristic amount extraction device 113 in FIG. 1, and thischaracteristic amount is the length in the horizontal directioncorresponding to the size of the character.

With the below-described image quality improvement unit, processes whichrestore missing pixels are primarily conducted, but in this case, thenumber of missing pixels differs depending on the character size. Inother words, large characters have a large number of missing pixels,while small characters have only a small number of missing pixels. Thus,by making a new evaluation value (called evaluation value F) out of thevalue found using the character size, which is this third characteristicamount, in addition to the above-described fourth and fifthcharacteristic amounts and evaluation values D and E, it becomespossible to conduct a more effective process in the image qualityimprovement unit.

This third characteristic amount is an average length of black runs atleast as long as a fixed length, and as explained above, this is a valueclose to the size of the character. The average length of black runs atleast as long as a fixed length is the length of the average of "blackruns at least as long as a fixed length" used when finding theabove-described second characteristic amount. In addition, thisevaluation value F can be expressed as follows:

    (evaluation value F)=(fourth and fifth evaluation values or evaluation values D and E)×(third characteristic amount)       (20)

FIG. 24 shows the order of processes in computing this evaluation valueF. This FIG. 24 is basically the same as the flowchart shown in FIG. 12.In FIG. 24, first each of the registers and counters are initialized(step S71), similar to the processes shown in the flowchart in FIG. 9.Next, the line data is prepared for n lines from the row indicated bythe row counter (step S72) because the orthogonal transformation isconducted on n rows, and the above-described fourth characteristicamount, fifth characteristic amount and third characteristic amount arecomputed (step S73). Furthermore, each time these fourth, fifth andthird characteristic amounts are computed, the evaluation value F iscomputed using above-described formula (20) (step S74). Next, thedetermination is made (step S75) as to whether the process of computingevaluation value F has been completed for the entire page of thedocument that is the object of processing, and if this process has notbeen completed, the row counter is incremented by n (step S76), and theprocesses in above-described steps S72 through S75 are conducted again.

In the above processes, each time the processes in step S73 arecompleted, the evaluation value F is computed in order that this kind ofimage quality determination process in this image quality computationunit 11 is conducted in parallel with the below-described image qualityimprovement processes in the image quality improvement unit or thebinary threshold value determination processes in the binary thresholdvalue determination unit. However, when the image quality computationprocess is first conducted as described for the entire page of thedocument that is the object of processing, following which all of theimage quality improvement processes or the binary threshold valuedetermination processes are conducted, in the process flow of FIG. 24all evaluation values F may be computed after the processes in step S73have been completed for an entire image. Similarly, computation of theevaluation values in n line data units may also be conducted for theabove-described fourth evaluation value and fifth evaluation value orevaluation values D and E, or the evaluation values may be computed forthe entire image.

Now, the following four patterns can be considered for this evaluationvalue F.

(a) (evaluation value F)=(fourth characteristic amount (fourthevaluation value))×(third characteristic amount)

(b) (evaluation value F)=(fifth characteristic amount (fifth evaluationvalue))×(third characteristic amount)

(c) (evaluation value F)=(evaluation value (evaluation value A) takingthe fifth characteristic amount into consideration in addition to thefourth characteristic amount)×(third characteristic amount)

(d) (evaluation value F)=(evaluation value (evaluation value B) takingthe fifth characteristic amount into consideration in addition to thefourth characteristic amount)×(third characteristic amount)

The above (a) is called evaluation value F1, (b) is called evaluationvalue F2, (c) is called evaluation value F3 and (d) is called evaluationvalue F4.

In this way, by using the third characteristic amount, which ischaracter size, it is possible to more efficiently conduct image qualityimprovement processes in the below-described image quality improvementunit.

With this embodiment, an example was shown in which the image qualitycomputation unit 11 is provided with a first characteristic amountextraction device 111, a second characteristic amount extraction device112, a third characteristic amount extraction device 113, a fourthcharacteristic amount extraction device 114, and a fifth characteristicamount extraction device 115. Furthermore, as noted to this point, eachevaluation value was computed, but as the image quality computationunit, it is possible to independently determine image quality relatingto "faintness" using either the first characteristic amount extractiondevice 111 or the fourth characteristic amount extraction device 114,and it is possible to independently determine image quality relating to"smudging" using either the second characteristic amount extractiondevice 112 or the fifth characteristic amount extraction device 115.

Next, when the image quality computations noted to this point areconducted, processes were described for the case wherein differentcharacter regions (e.g., the gothic character region Z1 wheredeterioration of the characters is difficult to produce, and the Minchocharacter region Z2 where deterioration is easily produced due tofaintness and the like) exist on the surface of the paper that is theobject of processing, as shown in FIG. 25(a).

When character regions of different types exist in this manner, in themethod whereby data is successively stored when the evaluation valuesare computed, as noted above, the problem arises that it is impossibleto obtain the correct evaluation values in the areas where the regionschange. This will be explained using FIGS. 25(b) and 25(c). In FIGS.25(b) and 25(c), the thick solid line L1 is the ideal evaluation valuesthat should be found in region Z1 and the thick solid line L2 is theideal evaluation values that should be found in region Z2. As can beseen from FIG. 25(b), in the method whereby the evaluation values arecomputed by successively storing data, the evaluation values that areactually found are not the correct evaluation values over an interval(the range indicated by w1 in the drawing) where the change is made fromregion Z1 to region Z2, shown by the thin solid line L3.

Hence with the present invention, a limit is provided to the range overwhich evaluation values are computed, making it possible to compute theappropriate evaluation values even when character regions of differencetypes exist. This method is described below.

In images wherein different types of characters exist, changes incharacter type within a row are rare, and most cases have changes inunits of rows. Accordingly, finding the evaluation values in units of acertain number of rows is also possible.

As one example of thus finding the evaluation values in units of acertain number of rows, the case of processing each row will bedescribed. In this case, first one row of text images is cut-out fromthe image, and evaluation values are found in the range of this cut-outrow. As this method of cutting out rows, a method exists whereby aprojection which is the cumulative value of the pixels in the rowdirection is found, and from this projection the determination is madethat the valleys are between rows so that the rows are thus cut-out.FIG. 26 shows an example of cumulative values for rows and between rows.As a still simpler method, a method exists wherein at the stage ofscanning the image, the absence or presence of pixels is detected andthe determination is made that the areas where there are no pixels arebetween rows, and the rows are thus cut-out. Furthermore, the evaluationvalues are found in units of cut-out rows with the above-describedmethods.

In addition, even in cases wherein character regions of differing typesexist, another method that enables computation of the appropriateevaluation values is to find the frequency of black and white reversalsin the line that is the object of processing, to reverse on lines wherethis frequency of black and white reversals reaches at least a presetnumber (the number that is necessary for finding evaluation values ispreset), and the evaluation values are found within this range. Forexample, suppose that the frequency of black and white reversals neededto find the evaluation values is 2000. At the point where the frequencyof black and white reversals reached 2000, the evaluation values wouldbe found in the range up to the point where this frequency of black andwhite reversals reached 2000. With this method, a process of cutting outrows is unnecessary.

By thus providing a limit to the range in which computation ofevaluation values is conducted, it becomes possible to find the optimumevaluation values in each of the regions (regions Z1 and Z2 in thiscase) as shown in FIG. 25(c) by computing the evaluation values in eachof the preset ranges. That is to say, the ideal evaluation values (thicksolid lines L1 and L2) that should be found in region Z1 and region Z2and the evaluation values (thin solid line L3) actually foundsubstantially coincide.

The process described here of finding the optimum evaluation values ineach region is an important process in the below-described process ofimproving image quality and process of determination the binarythreshold value suitable for an image. That is to say, the evaluationsvalues appropriate for each region out of these regions (for each typeof character) becomes important in the process of improving imagequality and the process of determination the binary threshold valuesuitable for an image.

Next, as a second aspect of the present invention, an image processingdevice will be described which is equipped with an image qualityimprovement unit (details of which are given hereinafter) that improvesimage quality using the results of the image quality computation unit 11described with respect to the above first aspect of the presentinvention.

FIG. 27 shows the composition of this second aspect of the presentinvention, and herein the image quality computation unit 11, the CPU 12,the RAM 13, the image input device 14, the image output device 15 andthe bus line 16 are the same as explained in FIG. 1, and in this FIG.27, in addition to these components an image quality improvement unit 21is also connected to the above-described bus line 16.

This image quality improvement unit 21 extracts sections where there isa possibility of pixels missing, and conducts an interpolation on thissection with pixels missing using the evaluation values computed by theabove-described image quality computation unit 11. Consequently, thisimage quality improvement unit 21 has a processing candidate extractiondevice 22 and a pixel processing device 23 as the major elements.Hereinafter, this processing candidate extraction device 22 and pixelprocessing device 23 will be described.

The processing candidate extraction device 22 extracts sections wherethere is a possibility of pixels missing, and consequently includes acharacteristic point extraction device 221 and a candidate determinationdevice 222. In addition, the pixel processing device 23 conducts aninterpolation of pixels on the sections where there is a possibility ofpixels missing, said sections having been extracted by the processingcandidate extraction device 22 and said interpolation conducted usingeach evaluation value computed by the above-described image qualitycomputation unit 11. Consequently, this pixel processing device has athreshold value computation device 231 and a character cut-out device232. Each of these components is described hereinafter.

The characteristic point extraction device 221 extracts the trace of thesections in which image quality has deteriorated. That is to say, as canbe seen from FIG. 2(b), the sections where faintness is produced arewhere pixels from one pixel to two pixels (or three pixels) lined up inthe vertical direction are convex in the horizontal direction. It isthese kinds of sections that are extracted. Specifically, at this timecharacteristic points having the six types of patterns shown in FIGS.3(a)-FIG. 3(f) are extracted.

The candidate determination device 222 performs determination of thedeteriorated sections from the positional relationship of thecharacteristic points extracted by the above-described characteristicpoint extraction device 221. As can be seen from FIG. 2(b), the sectionswhere image quality has deteriorated are primarily sections where thecharacteristic points oppose each other (face each other), such assection A in FIG. 2(b). However, some sections where characteristicpoints and black pixels that are not characteristic points oppose eachother, such as in section B. Thus this candidate determination device222 extracts these kinds of sections, and determines that said sectionsare sections with a possibility of image quality deterioration.

The combination of characteristic points of a deteriorated section isset as described below. However, this is intended to be illustrative andnot limiting.

(1) Three pixel characteristic points mutually facing each other (e.g.,FIG. 3(e) and FIG. 3(f)).

(2) Three pixel characteristic points and two pixel characteristicpoints facing each other (e.g. FIG. 3(c) and FIG. 3(f)).

(3) Three pixel characteristic points and one pixel characteristicpoint(s) facing each other (e.g. FIG. 3(a) and FIG. 3(f).

(4) Two pixel characteristic points mutually facing each other (e.g.,FIG. 3(c) and FIG. 3(d)).

(5) Two pixel characteristic points and one pixel characteristicpoint(s) facing each other (e.g. FIG. 3(a) and FIG. 3(d)).

(6) One pixel characteristic points mutually facing each other (e.g.,FIG. 3(a) and FIG. 3(b)).

(7) Three pixel characteristic points and black pixels other thancharacteristic points facing each other.

(8) Two pixel characteristic points and black pixels other thancharacteristic points facing each other.

(9) One pixel characteristic point(s) and black pixels other thancharacteristic points facing each other.

In these kinds of combinations, in the cases from (2) to (6) wherecharacteristic points mutually face each other, as a first conditionpixel interpolation can be considered necessary (processing candidate)if even some of the characteristic points are facing each other, buteven when characteristic points mutually face each other, in case (1)where three characteristic points mutually face each other, as a secondcondition pixel interpolation can be considered necessary (processingcandidate) only when two or more pixels face each other. Thisrelationship is shown in FIG. 28. FIG. 28(a) shows a case where theabove-described first condition is satisfied, and this is case (2) wherethree pixel characteristic points and two pixel characteristic pointsface each other, this example being one in which one pixel of each faceeach other. In addition, FIG. 28(b) shows a case where theabove-described second condition is satisfied, and this is case (1)where three pixel characteristic points mutually face each other, thisexample being one in which two pixels of each face each other. Inaddition, FIG. 28(c) is case (1) where three pixel characteristic pointsmutually face each other, but in this case two or more pixels do notface each other, so condition 2 is determined to not be satisfied.

In addition, in FIG. 28(a)-FIG. 28(c) pixel interpolation is conductedon the interval between facing pixels when the space between said facingpixels (indicated in the drawing by arrows) is a spacing within a presetnumber of pixels. In other words, the determination is made as to howmany white pixels there are between the facing pixels (black pixels),the threshold value of these white pixels is established, this whitepixel section is considered to be "faintness" when the this value isbelow the threshold value, and thus black pixels are interpolated intothis section. A detailed description of this is given hereinafter.

Next, the process will be described whereby interpolation of pixels isconducted in the above-described pixel processing device 23 on sectionswith pixels missing, said process performed using each evaluation valuecomputed by the above-described image quality computation unit 11.

This pixel processing device 23 has a threshold value computation device231 and a character cut-out device 232. This threshold value computationdevice 231 computes the threshold value at the time pixel interpolationis conducted (this threshold value will be called threshold value TH3)using each evaluation value computed by the above-described imagequality computation unit 11 as variables. This threshold valuecomputation device 231 is described hereinafter.

The sections which have been determined to be deteriorated sections bythe candidate determination device 222 in the processing candidateextraction device 22 have a space (white pixels) several pixels widebetween black pixels, such as sections A and B in FIG. 2(b). Inaddition, comparing FIG. 41(b) and FIG. 41(c) used in the paragraphsdescribing related art, this space is wider the larger the deteriorationof the image quality. Hence, the space in the deteriorated section canbe considered to be wider the larger the deterioration of image quality,and the threshold value TH3 is established as an index of the size ofspace up to which pixel interpolation is conducted. That is to say, whenimage quality is good, pixel interpolation is conducted even oversections without pixels in the original when the process of conductinginterpolation on the white pixel sections is performed. Hence, thethreshold value TH3 which agrees with image quality is computed in thethreshold value computation device 231 in order to prevent this kind oferroneous process.

The method of computing this threshold value TH3 in this threshold valuecomputation device 231 is described hereinafter.

The computation of threshold value TH3 is conducted for each of the 9combinations of deteriorated sections indicated in above-described cases(1) to (9). The reason for this is that a most suitable threshold valueexists for each combination indicated by above-described (1) to (9)depending on the combination. Thus, threshold value TH3 is computed withthe formula below.

    Threshold value TH3(n)=f(n) (x)                            (21)

Here, n takes on the values 1≦n≦9, and these numerical values from 1 to9 indicate the number inside the parentheses for the nine combinationsof deteriorated sections indicated by (1) to (9) above. In addition,f(n) (x) is a function (the method of creation of which is describedbelow) used to compute different threshold values for each combination,with x being a variable. This variable x is in this case the evaluationvalue. Accordingly, the above formula 21 can be expressed as:

    Threshold value TH3(n)=f(n) (evaluation value)             (22)

Here, evaluation value means each kind of evaluation value found by theimage quality computation unit 11. In this way, a threshold value TH3corresponding to each individual combination is established through thenine combinations of deteriorated sections indicated by (1) to (9).Furthermore, if the space in sections found by the candidatedetermination device 222 which have a possibility of image deteriorationis no greater than the threshold value TH3(n) found from formula (22),pixel interpolation is performed on these sections by the pixelprocessing device 21, while if this space is larger than the thresholdvalue TH3(n) found from formula (22), pixel interpolation is notperformed.

However, the function used to compute the above-described thresholdvalue TH3 performs computations as described below. Here, three types ofimages, namely, the original undeteriorated image such as shown in FIG.29(a), an image in which image quality has deteriorated somewhat such asshown in FIG. 29(b), and an unrepresented image in which image qualityhas deteriorated further--are prepared as standard images.

In the image in which image quality has deteriorated somewhat such as isshown in FIG. 29(b), extraction of the sections where image quality isthought to have deteriorated (the section indicated by C in the drawing)is performed by the candidate determination device 222. In addition,along with finding the type of combination of the characteristic pointsin the extracted section and the space between pixels, the determinationis made as to whether the section where image quality is thought to havedeteriorated is a section that has actually deteriorated by comparingsuch with the original image. Here, the type of combination of thecharacteristic points of the section indicated by C in the drawing istype (5) because two pixel characteristic points and one pixelcharacteristic points are facing each other, and in addition, thespacing in this case is two pixels.

This information is found for the entire image that is the object ofprocessing, and as shown in FIG. 29(b) all of the section wherein theimage quality has deteriorated somewhat and two pixel characteristicpoints and one pixel characteristic points are facing each other (theabove-described combination (5)) is cut-out, and the relationshipbetween the spacing in this deteriorated section is shown in FIG. 30along with the results of determining whether or not this deterioratedsection is actually a deteriorated section by comparing such with theoriginal image.

The graph in this FIG. 30 has on the horizontal axis the spacing (numberof pixels) of separation in the deteriorated section on the horizontalaxis, and on the vertical axis has the number of correct answers where acorrect answer is the case wherein the section is actually adeteriorated section as the result of determining whether the section isactually a deteriorated section by comparing such with the originalimage, and has the number of incorrect answers where an incorrect answeris the case wherein the section is not a deteriorated section as theresult of determining whether or not the section is actually adeteriorated section by comparing such with the original image, saidsection thus being an original white pixel section. In FIG. 30, the itemindicated by the solid line shows the number of correct answers, and theitem indicated by the dotted line shows the number of incorrect answers.

From the graph it can be seen that correct answers (deterioratedsections) are more prevalent up to a spacing of three pixels, whileincorrect answers (originally white pixel sections) are more prevalentwhen the spacing becomes 4 pixels (in FIG. 30, the shaded area is thesection where the number of correct answers is larger.) Accordingly, inthis case threshold value TH3(5) is 3. That is to say, inabove-described combination (5), the section is considered adeteriorated section up to a spacing of 3, indicating that no problemswill arise if this space is interpolated. Conversely, when the spacingis 4 or greater, the determination is made that this is a space whichexisted originally, indicating that it is better to not perform theinterpolation process.

Similarly, suppose that the threshold value TH3(5) is 6 as a result offinding a graph showing the spacing in deteriorated sections in the casewhere two pixel characteristic points and one pixel characteristicpoints face each other in further deteriorated images, and showing theresults of determining whether this deteriorated section is actually adeteriorated section by comparing such with the original image.

In addition, suppose that the evaluation value of the somewhatdeteriorated image shown in FIG. 29(b) (any of the several evaluationvalues computed with the first aspect of the present invention may beused) is 0.49, and further suppose that the evaluation value of thefurther deteriorated image (here also, any of the several evaluationvalues computed with the first aspect of the present invention may beused) is 0.99.

Now suppose that the above-described function f(n) (evaluation value)is, for example, expressed as a first degree function of the type ax+b.Because in this case the variable x is the evaluation value, the resultis:

    Threshold value TH3(n)=a x evaluation value+b              (23)

In this case, because n=5, the result is:

    Threshold value TH3(5)=a x evaluation value+b              (24)

Finding a and b by substituting into this formula 24 the threshold valueTH3(5)=3 and evaluation value 0.49 and the threshold value TH3(5)=6 andevaluation value 0.99, said values found as described above, it is foundthat a=6 and b=0.06, so that the function in the case "(5) Two pixelcharacteristic point and one pixel characteristic point facing eachother" in the image that is the object of processing is determined. Inthis way, the functions are determined for each of the types ofcombinations from (1) to (9).

In this way, the functions are set for each of the types of combinationsfrom (1) to (9). That is to say, in this case a line is obtained withthe evaluation value as the variable, such as in FIG. 31, and throughthis the other threshold value(s) TH3(n) are found.

Because the threshold value is an integer, it is also possible to use avalue which has been made into an integer value through rounding off thepart after the decimal in the value found from the above-describedfunction. Furthermore, the determination as to whether or not tointerpolate pixels is made from the relationship between the detectedspacing and this threshold value TH3. That is to say, interpolation ofpixels is performed on a detected spacing if this spacing is no greaterthan the threshold value TH3, and interpolation of pixels is notperformed on a detected spacing if this spacing is larger than thethreshold value TH3 because such a space is considered to be originallywhite.

FIG. 32 shows the flowchart for the above process.

In FIG. 32, each type of counter and register are first initialized(step S81). This process is the same as in step S11 of the flowchart inFIG. 7. Next, six lines of data are prepared starting with the rowindicated by the row counter (step S82). In this case, when three pixelcharacteristic points mutually facing each other as shown in FIG. 28, atleast six lines of data are necessary in order to determine whether ornot condition 2, which requires at least two of these pixels on eachside to be facing each other, has been satisfied.

Furthermore, computation of each evaluation value is performed on eachline as described in the first aspect of the present invention above.The series of processes from step S81 to step S85 is similar to theprocess flowchart in FIG. 12.

On the other hand, an image improvement process is performed in parallelwith the process in step S83. That is to say, detection of thecharacteristic point combination in (1) to (9) above is conducted, andthe determination of the spacing is also conducted (step S86). Then, thethreshold value TH3(n) is found from the number of the characteristicpoint combination from (1) to (9) above and the evaluation values (stepS87), and this threshold value TH3(n) and the spacing detected in stepS86 are compared (step S88) and an interpolation of pixels is performedif the threshold value TH3(n) is at least as great as the spacing (stepS89).

The reason a character cut-out device 232 is provided in the pixelprocessing device 23 of the image quality improvement unit 21 describedin FIG. 27 is to try to perform the interpolation process afterextracting characters one at a time because there are cases whereinerroneous pixel interpolation processes are conducted between adjacentcharacters, for example, such as is shown in FIG. 33. In other words, asshown in FIG. 33, where there is a section s where adjacent charactersare extremely close to each other, there is the possibility that thissection s could be determined to be a section where pixel interpolationis necessary and an erroneous process could be performed, so in order toprevent this the characters are cut-out one character at a time, and theinterpolation process is conducted within the range of the cut-outcharacter. Through this, prevention of erroneous pixel interpolationprocesses between adjacent characters can be achieved.

FIG. 34(a)(I)-FIG. 34(b)(III) show concrete examples of images restoredusing the above-described image restoration process. FIG. 34(a)(I) isthe image shown in FIG. 41(b) which has been copied once, while FIG.34(b)(I) is the image shown in FIG. 41(c) which has been copied twice.By employing the above-described processes on these images, the imagesbecome such that the "faintness" sections are substantially restored, asshown in FIG. 34(a)(II) and FIG. 34(b)(II). In addition, when a processthat eliminates unevenness in the image (a smoothing process) is alsoemployed in addition to the above processes, an image which appears evenbetter results, as shown in 1.

When these kinds of processes are employed, the recognition rate in thecharacter recognition device is improved, in the case of the image thathas been copied once as shown in FIG. 34(a) (I-III), from 93.4% for theimage in FIG. 34(a)(I) to 97.1% for the image in FIG. 34(a)(II), and inaddition, is improved in the case of the image which has been copiedtwice as shown in FIG. 34(b) (I-III), from 82.3% for the image in FIG.34(b)(I) to 93.1% for the image in FIG. 34(b)(II).

In this way, by determining the threshold value TH3 in accordance withthe image quality, pixel interpolation is performed on a detected spacewhen said space is smaller than this threshold value, and pixelinterpolation is not performed on a detected space when this space islarger than the threshold value and therefore considered to originallybe white space. Consequently, it is possible to prevent erroneousprocesses which perform pixel interpolation even on sections withoutpixels, and it becomes possible to accurately conduct restorationprocesses only on sections deteriorated by faintness or the like.

Accordingly, when this image improvement device is applied to digitalcopiers and character recognition devices and the like, the results aredramatic. In particular, the results are even more noticeable in digitalcopiers. That is to say, a digital copier is a device used to faithfullyreproduce an image, and if the original document is faint, a copy isproduced which is also faint. However, by applying the image processingdevice of the present invention, it becomes possible to perform an imagequality improvement process according to quality when it is determinedthat the image quality is poor. In other words, because a copy isproduced in which the faint sections have been restored (through pixelinterpolation or the like) when the original document is faint, aduplicate document is obtained in which the quality is better than inthe original document. In addition, because the processes are performedwhile making determinations about image quality, extra processes are notadded to documents with good image quality, so that there are no illeffects of such. Furthermore, even in character recognition devices, bydetermining the image quality prior to the character recognition processand then conducting image quality improvement processes in accordancewith quality when it is determined that image quality is poor, it ispossible to greatly increase the character recognition rate.

In this second aspect of the present invention, the evaluation valuesused as variables in finding this threshold value TH3 can be any of thevarious evaluation values computed in the above-described first aspectof the present invention. That is, the evaluation values can beevaluation values C1 to C4 which are found by multiplying the thirdcharacteristic amount by the first evaluation value which is the firstcharacteristic amount, the second evaluation value which is the secondcharacteristic amount, and evaluation values which taken intoconsideration the first characteristic amount and the secondcharacteristic amount (e.g. evaluation value A which is found bysubtracting the second characteristic amount from the firstcharacteristic amount, or evaluation value B which is found by addingthe second characteristic amount to the first characteristic amount); orevaluation values F1 to F4 which are found by the third characteristicamount by the fourth evaluation value which is the fourth characteristicamount, the fifth evaluation value which is the fifth characteristicamount, or evaluation values which take into consideration the fourthcharacteristic amount and the fifth characteristic amount (e.g.evaluation value D which is found by subtracting the fifthcharacteristic amount from the fourth characteristic amount, orevaluation value E which is found by adding the fifth characteristicamount to the fourth characteristic amount). However, better results areobtained by using either evaluation values C1 to C4 or evaluation valuesF1 to F4.

In addition, with this second aspect of the present invention, thefunction in formula (22) used to find threshold value TH3 wasillustrated as a first degree function, but it is possible to use ann-degree function, for example, by preparing a plurality of deterioratedimages. Furthermore, in order to simplify the process, the function informula (22) can be a fixed value instead of a function using theevaluation value as a variable.

Next, devices for determining the optimum binary threshold valuesuitable for the character image are described as a third aspect of thepresent invention.

A multitude of methods have been proposed from before as methods ofbinary coding, but these conventional methods cannot necessarily be saidto be binary coding methods which are suitable for character images. Inother words, there are cases where faintness and smudging are producedafter binary coding. Hence, with the present aspect of the presentinvention, a method is described for determining the binary thresholdvalue using the evaluation values found by the above-described imagequality computation unit 11.

FIG. 35 is a block diagram showing the structure of this third aspect ofthe present invention with the first aspect of the present invention.This structure is similar to the structure shown in FIG. 1 in which abinary threshold value determination unit 31 has been provided.

The contents of the processes of this binary threshold valuedetermination unit 31 are described below with reference to theflowchart in FIG. 36.

First, the image that is the object of processing is input as multiplevalues (step S91). Here, it is supposed that the input is in 16gradations, with 0 being the white gradation and 15 being the blackgradation. Next, a dummy binary threshold value for binary coding isdetermined (step S92). The central value in the 16 gradations is 7 or 8,and here, let it be supposed that the determination range of the dummybinary threshold value is changeable within the range of gradations from5 to 11, and suppose that the initial value of the dummy binarythreshold value is 8. The reason the determination range of the dummybinary threshold value is taken to be from 5 to 11 is because in generalthreshold values suitable for binary coding exist in this range in thecase of 16 gradation input.

Furthermore, binary coding is performed using this dummy binarythreshold value which has been determined in this manner (step S93).After this binary coding process, computation of the evaluation value(s)is conducted (step S94). Any of the above-described methods may be usedhere as the method of computing the evaluation value, but here thedescription will be provided for an example that uses the evaluationvalue (evaluation value E) found by adding the fifth characteristicamount to the fourth characteristic amount as described in FIG. 23. Inthis case, the place where the evaluation value E is a minimum is theplace where the best image quality is obtained, as has been noted above.Accordingly, it is preferable to set the place where this evaluationvalue E is a minimum as the binary threshold value. Below, thedetermination of a threshold value wherein the evaluation value Ebecomes a minimum is described. FIG. 37 is a graph showing therelationship between the binary threshold value TH1 of the image that isthe object of processing and the error rate (indicated by the dottedline) and the evaluation value E (indicated by the solid line), wherethe evaluation value E is a value found by adding the fifthcharacteristic amount to the fourth characteristic amount. In this FIG.37, the "+" symbols attached to the solid line which indicates theevaluation value E indicate that the value of "faintness" is at least asgreat as the value for "smudging", while the "-" symbols attached to theline indicates all other states. The order of processes is describedbelow with reference to FIG. 38.

In FIG. 38, first the initial value of the dummy binary threshold valueTH1 is set to 8 (step S101), following which the evaluation value isfound at an initial value of 8 for this binary threshold value TH1, andthe determination is made as to the size of the faintness and smudgingat this time, i.e., the determination is made as to whether or not"faintness≧smudging" (step S102). At this determination, in this casethe determination is made that the binary threshold value that should befound is smaller than 8 because smudging is greater than faintness, ascan be seen from FIG. 37, so the dummy threshold value TH1 is set to 6(step S103). Furthermore, the evaluation value is found at this dummybinary threshold value of 6, and the determination is made as to thesize of the faintness and smudging at this time, i.e., the determinationis made as to whether or not "faintness≧smudging" (step S104). At thisdetermination, in this case "faintness smudging", so consequently thedummy binary threshold value TH1 is set to 7 (step S105). Furthermore,the evaluation value is found at this dummy binary threshold value TH1of 7, and here "faintness≧smudging." Hence, because the dummy binarythreshold value TH1 of 7 is close to faintness and a value TH1 of 8 isclose to smudging, either of these can be the optimum binary thresholdvalue. Thus the binary threshold value which corresponds to the smallerevaluation value out of these two evaluation values (or the threeevaluation values that have been found) becomes the optimum binarythreshold value for 16 gradations, so in this case, this optimum binarythreshold value is TH1=7. In other words, the binary threshold value TH1is determined to converge to 7 (step S106). In this way, in this case itis possible to determine the binary threshold value that should be foundthrough three dummy binary threshold value determination processes andevaluation value computations.

Here, the above description has been for the case using evaluation valueE found by adding the fifth characteristic amount to the fourthcharacteristic amount, but in cases using evaluation values found bysimple subtraction or simple addition such as is shown in FIG. 6(a) orFIG. 8(a), by setting a point as a standard for the evaluation value(e.f., the dashed line showing in these figures), it becomes possible toconduct processes similar to that described above. In addition, thenumber of gradations is not restricted to 16, and the convergence methodfor determining the binary threshold value is not restricted to theabove-described method either. In addition, when this is applied to ascanner, because input of multiple values of data requires time for datatransfer, determining of the binary threshold value and binary inputfrom the scanner may be conducted by inputting a binary value from thescanner at the dummy binary threshold value, and by successivelychanging the binary threshold value while performing the convergencedetermination and evaluation value computations with a personalcomputer. In addition, after multiple values are input in some regionsand the binary threshold value has been determined, input of all binarydata may be conducted through this binary threshold value. Furthermore,it is also possible to divide the input image into several regions(e.g., units of character rows) and to conduct determination of thebinary threshold value in each of these divided regions.

As described above, with the third aspect of the present invention theplace where the best image quality is obtained can be determined as thebinary threshold value, and consequently by applying this third aspectof the present invention to a copy machine or a scanner or the like, itis possible to obtain high quality images with little faintness andsmudging. In addition, by using such as a pre-process for OCRs,supposing that the default binary threshold value were TH1=8conventionally as in the example shown in FIG. 37, it is possible to setthis value to TH1=7 at which a high recognition rate is automaticallyobtained, and thus it is possible to obtain a high recognition rate.

FIG. 39 is a drawing showing the structure of an image processing devicewhich, in addition to performing binary coding through a binarythreshold value determination method such as described above, is alsocapable of a process to improve images which have been deteriorated byfaintness and smudging, or the like. The basic structure here is thesame as that of FIG. 27, and to this structure in FIG. 27 the binarythreshold value determination unit 31 described in the above thirdaspect of the present invention has been provided. In other words, thisdevice has a structure combining an image quality computation unit 11,an image quality improvement unit 21 and a binary threshold valuedetermination unit 31.

The rough processes of this image processing device consist ofconducting binary coding by determining the binary threshold value usingthe evaluation values found by the above-described image qualitycomputation unit 11, and conducting a process to improve image qualityusing the above-described evaluation values with respect to the imageafter binary coding. Detailed descriptions have been providedhereinbefore of the evaluation value computation process in thisprocess, the binary threshold value determination process using thisevaluation value(s), and the process of improving image quality usingthe above-described evaluation values. Consequently, a description ofthe order of these processes is omitted here.

The process of performing binary coding by determining a binarythreshold value using the evaluation values and the process of improvingimage quality using the above-described evaluation values with respectto the image following binary coding may be conducted by firstperforming binary coding by determining the binary threshold value forthe entire surface of the image that is the object following which imageimprovement may be conducted, or these processes may be conducted inparallel (the process of conducting binary coding by determining thebinary threshold value is performed first).

In this way, binary coding is conducted by determining a binarythreshold value using the evaluation values found by the image qualitycomputation unit 11, and a process is conducted to improve image qualityby using the above-described evaluation values with respect to the imagefollowing binary coding, and through this it is possible to set theplace where the best image quality is obtained as the binary thresholdvalue. Consequently, it is possible to obtain a high quality image withlittle faintness and smudging in the binary coding gradations. Inaddition, in cases where the image quality is determined to be poor, itis possible to conduct a process to improve image quality. In otherwords, in a case where the original document is plagued with faintness,even if the binary threshold value is determined optimally, this doesnot mean that the faintness subsides. Hence, because copying can beconducted which restores the points of faintness (through pixelinterpolation or the like) by conducting a process to improve imagequality further, a copy of the document can be obtained which has betterquality than the original document. In addition, because the process isconducted while determining image quality, there is no addition of extraprocesses when the quality of the document is good, so that there are noill effects of such. Furthermore, even in character recognition devices,by determining the image quality prior to the character recognitionprocess and then conducting image quality improvement processes inaccordance with quality when it is determined that image quality ispoor, it is possible to greatly increase the character recognition rate.

In addition, in the embodiments described thus far, the descriptionshave primarily been of devices as applied to the Japanese language, butapplication to alphanumerics (αN) is also possible by employing thefollowing alterations.

First, the sections noting the first aspect of the present invention arehandled as follows.

1) The formula for computing the first characteristic amount shown byformula (1) above is changed as shown below.

First characteristic amount=(frequency of appearance of one pixelcharacteristic points)/(frequency of black and white reversals)

Here, one pixel characteristic points are the characteristic pointsshown in FIGS. 3(a) and 3(b). The reason for this change is that in thecase of alphanumerics, "faintness" sections are essentially all onepixel convex parts.

2) In formula (2) above, i.e. the formula for computing the secondcharacteristic amount, which is:

Second characteristic amount=Number of black runs longer than thespecified length/Total number of black runs.

The method of computing the threshold value TH2 for counting the numberof black runs is changed. The threshold value TH2 in the first aspect ofthe present invention was found based on the number of pixels in ahorizontal line close to the size of the character, but alphanumericcharacters have low linearity horizontally in comparison to the Japaneselanguage, making it difficult to determine the size of the characterfrom a horizontal line. In addition, the determination can beaccomplished with more accuracy by determining the size of alphabeticcharacters based on the thickness of a vertical line as the standard.From this point, the formula is taken to be:

    Binary threshold value TH2=α×(thickness of vertical line in character)

Here, α is in the range 3.0 to 4.0. This is because the size ofalphabetic characters and the like is usually about 3 to 4 times thethickness of the vertical line in the character, and from experiments,the value α=3.4 was found to be suitable. In other words, it can bedetermined that the length of the series of pixels of a character in thehorizontal direction is about 3.4 times the thickness of the verticalline in the character. Here, the thickness of the vertical line of thecharacter is taken as the average of the length of black runs at leastsix pixels long (corresponding to about one-third to one-fourth of thesize of the character during 300 DPI input.)

In addition, the sections noting the second aspect of the presentinvention are handled as follows.

3) The pixel processing device 23 is changed

In alphanumerics, touching of an adjacent character near the base lineof a character is relatively common, and because this does not cause adrop in the recognition rate even when there is a good deal of touchingnear the base line, and in addition because this does not make thecharacter look bad as viewed by the naked eye, a threshold value in theinterpolation process near the baseline and a threshold value in theinterpolation process other than near the base line are determinedseparately.

This is described with reference to FIG. 40.

FIG. 40 shows an example wherein the letters "bri" have been written,and in the case of letters, as explained above, there is no drop in therecognition rate even when there is a good deal of touching of adjacentcharacters near the base line 301, and in addition this does not makethe characters look bad when viewed by the naked eye. In contrast, insections other than near the base line 301, for example when the sectionindicated by 302 in FIG. 40 merge, in this case the letters appear toform the letter "n". Accordingly, the threshold value TH3 is set smallfor sections other than near the base line 301, and to the extentpossible, interpolation processes are not conducted. Near the base line301, it is acceptable if the threshold value TH3 is somewhat large.

Detection of the base line 301 is accomplished when this kind of processis conducted by investigating a line about two pixels below a sectionwhich is a candidate for image deterioration, e.g. point Q1 in thesection indicated by 303 in FIG. 40, and by determining that this pointQ1 is on the base line if no black pixels exist within about 10 pixelsto the left and right (a total of 20 pixels) centered at above-describedQ1 on this line 301. The aforementioned 20 black pixels correspondroughly to the size of a single character.

Through performing this kind of process, it is possible to conduct pixelrestoration even with alphanumeric characters.

With each of the above-described aspects of the present invention,examples were described wherein determination of threshold values andthe like was conducted for cases where the character resolution was 300DPI, but the present invention can also be applied to other resolutions.For example, each threshold value and the like may be multiplied bytwo-thirds in the case of a resolution of 200 DPI.

In addition, with each of the above-described aspects of the presentinvention, characters were the object of processing, and consequently,discrimination between drawings and charts and noise is determined bythe length of the string of black pixels. In other words, the length ofthe string of black pixels in charts and drawings is normallyconsiderably longer than that of characters, while conversely that ofnoise is shorter. Accordingly, charts and drawings and noise can bedetermined from this length, so that such do not become the object ofprocessing. Charts and the like are not the object of processing of thepresent invention at the point of computing evaluation values, but thetechnology of tracing pixels in deteriorated sections and restoring suchcan be applied to this case also.

In addition, applications of the present invention to color are alsopossible. That is to say, it is possible to apply the present inventionto such cases by conducting the processes described in these embodimentson each RBG that has been binary coded.

The image processing method of the present invention, as describedabove, includes an image quality computation process wherein acharacteristic amount may be extracted to determine the image quality ofthe image data entered by an image input device, and wherein thecharacteristic amount may be computed as an evaluation value, and theimage quality may be determined by the evaluation value obtained by theimage quality computation process, resulting in more objectiveevaluation of the image quality, enabling more effective development ofcopy machines than does the traditional subjective evaluation. Moreover,using the evaluation values thus obtained, capabilities of copy machinesare assessed more objectively and accurately.

Moreover, several patterns of pixel characteristics may be provided, tobegin with, as characteristic points and the first characteristic amountwhich is the ratio of the frequency of the appearance of saidcharacteristic points in the processing lines and the frequency ofreversals of black pixels and white pixels may be computed to be used asan index for faintness, and the image quality may be determined usingthe first evaluation value; thus, deterioration due to faintness of theimage is determined accurately.

Furthermore, the average length of a continuous string of black pixelsnearly equivalent to the size of a character may be obtained, and thenumber of continuous strings of black pixels longer than the averagecontinuous string of black pixels nearly equivalent to the size of acharacter may be computed. Then a second characteristic amount which isthe ratio of the number of continuous strings of black pixels longerthan the average length and one-half of the number of reversals of blackpixels and white pixels may be computed to be used as an index forsmudging; thus, deterioration due to smudging may be determinedaccurately.

In addition, the first characteristic amount to be used as an index forfaintness and the second characteristic amount to be used as an indexfor smudging may be computed, and evaluation values based on these twotypes of characteristic amounts may be obtained, enabling more accuratedetermination of deteriorated sections of the image than when only thefirst or the second evaluation value is used. Especially in the casewhen the evaluation value is obtained by adding the first characteristicamount and the second characteristic amount, the binary threshold valuewhich results in the best image quality can be determined easily bydefining the threshold value corresponding to the part in which theevaluation value obtained by the two evaluation values becomes thesmallest as the desired threshold value. In this manner, the binarythreshold determining process may be simplified.

Moreover, a third characteristic amount representing the size of thecharacter may be obtained in addition to said first characteristicamount. By adding the third characteristic amount representing the sizeof the character, the restoration process of the deteriorated sectiondue to missing pixels, which increases in proportion to the size of thecharacter, can be performed more effectively. Thus, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process.

Furthermore, a third characteristic amount representing the size of thecharacter may be obtained in addition to said second characteristicamount. By adding the third characteristic amount, more effectiveexecution of the image quality improvement process is enabled in asimilar manner as above.

In addition, a third characteristic amount representing the size of thecharacter may be obtained in addition to said first and said secondcharacteristic amount. In a similar manner as above, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process. Moreover, by considering the thirdcharacteristic amount after determining the image quality using twotypes of characteristic amounts, the first and the second characteristicamounts, a more effective execution of the image quality improvementprocess is enabled.

Moreover, orthogonal transformation of the input image data intofrequency regions may be performed, and a fourth characteristic amountto be used as an index for faintness may be computed by focusing on thehigh frequency component after orthogonal transformation. Then, bydetermining the image quality using a fourth evaluation value, which isthe fourth characteristic amount computed above, deteriorated sectionsdue to faintness of the image may be determined accurately. Theorthogonal transformation also enables determination of the direction ofthe deterioration.

Furthermore, orthogonal transformation of the input image data intofrequency regions may be performed, and a fifth characteristic amount tobe used as an index for smudging may be computed by focusing on the lowfrequency component after orthogonal transformation. Then, bydetermining the image quality using a fifth evaluation value, which isthe fifth characteristic amount computed above, deteriorated sectionsdue to smudging of the image may be determined accurately.

In addition, a fourth characteristic amount to be used as an index forfaintness and a fifth characteristic amount to be used as an index forsmudging, both obtained in the frequency regions, may be computed, andevaluation values based on these two types of characteristic amounts maybe obtained, enabling more accurate determination of deterioratedsections of the image than when only the fourth or the fifth evaluationvalue is used. Especially in the case when the evaluation value isobtained by adding the fourth characteristic amount and the fifthcharacteristic amount, the binary threshold value which results in thebest image quality can be determined easily by defining the thresholdvalue corresponding to the part in which the evaluation value obtainedby adding two evaluation values becomes the smallest as the targetbinary threshold value. In this manner, the binary threshold determiningprocess is simplified.

Moreover, a third characteristic amount representing the size of thecharacter may be obtained in addition to said fourth characteristicamount. By adding the third characteristic amount representing the sizeof the character, the restoration process of the deteriorated sectiondue to missing pixels, which increases in proportion to the size of thecharacter, can be performed more effectively. Thus, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process.

Furthermore, a third characteristic amount representing the size of thecharacter may be obtained in addition to said fifth characteristicamount. By adding the third characteristic amount, more effectiveexecution of the image quality improvement process may be enabled in amanner similar to the above.

In addition, a third characteristic amount representing the size of thecharacter may be obtained in addition to said fourth and said fifthcharacteristic amount. In a similar manner as above, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process. Moreover, by considering the thirdcharacteristic amount after determining the image quality using twotypes of characteristic amounts, the fourth and the fifth characteristicamounts, a more effective execution of the image quality improvementprocess is enabled.

Moreover, if different regions exist on an original to be processed, arestriction may be imposed on the ranges where said characteristicamount is extracted, and the evaluation value may be computed byextracting a characteristic amount from each range; thus, computation ofan evaluation value matching each range is enabled even when differenttypes of range, such as gothic fonts and mincho fonts, exist next toeach other. Since an evaluation value matching each range is necessaryin performing the image quality improvement process and the binarycoding process, computation of an evaluation value matching each rangebecomes very effective.

Furthermore, by providing an image quality computation process toextract a characteristic amount to determine the image quality of theimage data entered by an image input device, and to compute anevaluation value which is the extracted characteristic amount; and byproviding an image quality improvement process to determine, from thecharacteristic amounts, candidates for the image quality improvementprocess by extracting sections which have the possibility ofdeteriorated image quality, and to execute the image quality improvementprocess on candidates for image quality improvement processing by usingthe evaluation values obtained by said image quality computationprocess, accurate determination of deteriorated image sections isenabled. Moreover, missing pixels in the deteriorated section aredetermined and the interpolation process of pixels is enabled only whenpixels are missing due to deterioration; thus, accurate restoration ofdeteriorated pixels is realized. Therefore, by applying this to a copymachine, when an original has faintness, for example, the presentinvention produces a higher quality copy than the original by restoringthe faint sections in the original; though, in general, the copiedoriginal has the same level of faintness or worse than the original.Moreover, the present invention executes processes while determiningimage quality, but it has no effect on an original with a high qualityimage, leaving the high quality original as it is. Also, the presentinvention can be applied to a character recognition device to obtain ahigh rate of recognition.

In addition, a processing candidate extraction process to extractcandidates for image quality improvement processing, and pixelprocessing to interpolate pixels in executing image quality improvementon processing candidates extracted by the processing candidateextraction process may be provided, and only sections with deterioratedimage quality may be extracted, where the image quality improvementprocess may be performed using evaluation values computed by said imagequality computation unit. Thus, the image quality improvement processmay be performed only when image quality is poor, preventing erroneousoperations such as addition of new pixels in a section which wasinitially blank.

Moreover, by providing a characteristic point extraction process todetect and extract characteristic points generated by deterioration in asection with deteriorated image quality, and a candidate determinationprocess to determine candidates for image quality improvement using thepositional relationship of the characteristic points extracted by thecharacteristic point extraction process, characteristic points frompixels facing each other across an empty space produced by deteriorationmay be extracted and, using the positional relationship of the extractedcharacteristic points, determination of candidates for the imageimprovement process is enabled. Hence, deteriorated sections where theimage improvement process is to be performed are extracted effectivelyand accurately.

Furthermore, a threshold computation process may be provided which,using the evaluation value obtained by said image quality computationprocess, obtains a threshold value from a function with the evaluationvalue as a variable, compares said threshold value with the interval onwhich interpolation of pixels is performed, and determines whether toperform the interpolation process of pixels based on the results of thecomparison. Hence, the interpolation process may be performed only ondeteriorated sections due to missing pixels according to the degree ofimage quality deterioration, preventing erroneous operations such asaddition of new pixels in a section which was initially blank.

Furthermore, a character cutting-out process may be provided and pixelinterpolation may be performed within the region of characters which arecut out by the character cutting-out process to improve image quality.Hence erroneous pixel interpolation operation between adjacentcharacters may be prevented without fail.

In addition, an image quality computation unit to extract acharacteristic amount to determine the image quality of the image dataentered by an image input device and to compute the characteristicamount as an evaluation value, and a binary threshold determinationprocess to determine the binary threshold value for the image to beprocessed using the evaluation value obtained by the image qualitycomputation process may be provided. Hence, a binary threshold valuesuitable for the character may be determined, minimizing faintness andsmudging of the image after the binary coding process.

Moreover, the binary threshold value determination process may define athreshold value which, in case the evaluation value is obtained byadding the first evaluation value and the second evaluation value amongthe evaluation values obtained by said image quality computationprocess, corresponds to the section making the evaluation value,obtained by adding the two evaluation values, minimum as the targetbinary threshold value. Hence the binary threshold value producing thebest image quality may be determined easily and the binary thresholdvalue determination process may be simplified.

Furthermore, an image quality computation unit to extract acharacteristic amount to determine the image quality of the image dataentered by an image input device and to compute the characteristicamount as an evaluation value, a binary threshold determination processto determine the binary threshold value for an image to be processedusing an evaluation value obtained by the image quality computationunit, and an image quality improvement process to determine candidatesfor the image quality improvement process by extracting sections whichhave the possibility of image quality deterioration based on thecharacteristics and to perform the image quality improvement process oncandidates for image quality improvement processing using the evaluationvalue obtained by said image quality computation process may beprovided, enabling determination of a binary threshold value suitable tothe character. Hence, faintness and smudging of images after the binarycoding process may be minimized, and the deteriorated section may befurther improved. Application of the present invention to copy machines,character recognition devices, and the like will produce good results.

In addition, the image processing device of the present invention mayinclude an image quality computation unit wherein a characteristicamount may be extracted to determine the image quality of the image dataentered by an image input device, and wherein the characteristic amountmay be computed as an evaluation value, and the image quality may bedetermined by the evaluation value obtained by the image qualitycomputation unit, resulting in more objective evaluation of the imagequality, enabling more effective development of copy machines than doestraditional subjective evaluation. Moreover, using the evaluation valuesthus obtained, capabilities of copy machines may be assessed moreobjectively and accurately.

Moreover, several patterns of pixel characteristics may be provided, tobegin with, as characteristic points; and the first characteristicamount, which is the ratio of the frequency of the appearance of saidcharacteristic points in the processing lines and the frequency of thereversals of black pixels and white pixels, may be computed to be usedas index for faintness; and image quality may be determined using thefirst evaluation value. Thus deterioration due to faintness of the imagemay be determined accurately.

Furthermore, the average length of a continuous string of black pixelsnearly equivalent to the size of a character may be obtained, and thenumber of continuous strings of black pixels longer than the averagecontinuous string of black pixels nearly equivalent to the size of acharacter may be computed. Then a second characteristic amount, which isthe ratio of the number of continuous strings of black pixels longerthan the average length and one-half of the number of reversals of blackpixels and white pixels, may be computed to be used as an index forsmudging. Thus deterioration due to smudging may be determinedaccurately.

In addition, the first characteristic amount to be used as an index forfaintness and the second characteristic amount to be used as an indexfor smudging are computed, and evaluation values based on these twotypes of characteristic amounts may be obtained, enabling more accuratedetermination of deteriorated sections of the image than when only thefirst or the second evaluation value is used. Especially in the casewhen an evaluation value is obtained by adding the first characteristicamount and the second characteristic amount, the binary threshold valuewhich results in the best image quality can be determined easily bydefining the threshold value corresponding to the part in which theevaluation value obtained by the two evaluation values becomes thesmallest as the target binary threshold value. In this manner, thebinary threshold determining process may be simplified.

Moreover, in a third characteristic amount representing the size of thecharacter may be obtained in addition to said first characteristicamount. By adding the third characteristic amount representing the sizeof the character, the restoration process of the deteriorated sectiondue to missing pixels, which increases in proportion to the size of thecharacter, can be performed more effectively. Thus, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process.

Furthermore, a third characteristic amount representing the size of thecharacter may be obtained in addition to said second characteristicamount. By adding the third characteristic amount, more effectiveexecution of the image quality improvement process is enabled in amanner similar to the above.

In addition, a third characteristic amount representing the size of thecharacter may be obtained in addition to said first and said secondcharacteristic amount. In a similar manner as above, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process. Moreover, by considering the thirdcharacteristic amount after determining the image quality using twotypes of characteristic amounts, the first and the second characteristicamounts, a more effective execution of the image quality improvementprocess is enabled.

Moreover, orthogonal transformation of the input image data intofrequency regions may be performed, and a fourth characteristic amountto be used as an index for faintness may be computed by focusing on thehigh frequency component after orthogonal transformation. Then, bydetermining the image quality using a fourth evaluation value, which isthe fourth characteristic amount computed above, deteriorated sectionsdue to faintness of the image may be determined accurately. Theorthogonal transformation also enables determination of the direction ofthe deterioration.

Furthermore, in orthogonal transformation of the input image data intofrequency regions may be performed, and a fifth characteristic amount tobe used as an index for smudging may be computed by focusing on the lowfrequency component after orthogonal transformation. Then, bydetermining the image quality using a fifth evaluation value, which isthe fifth characteristic amount computed above, deteriorated sectionsdue to smudging of the image may be determined accurately.

In addition, in the fourth characteristic amount to be used as an indexfor faintness and the fifth characteristic amount to be used as an indexfor smudging, both obtained in the frequency regions, may be computed,and evaluation values based on these two types of characteristic amountsmay be obtained, enabling more accurate determination of deterioratedsections of the image than when only the fourth or the fifth evaluationvalue is used. Especially in the case when an evaluation value isobtained by adding the fourth characteristic amount and the fifthcharacteristic amount, the binary threshold value which results in thebest image quality can be determined easily by defining the thresholdvalue corresponding to the part in which the evaluation value obtainedby adding two evaluation values becomes the smallest as the desiredthreshold value. In this manner, the binary threshold determiningprocess may be simplified.

Moreover, a third characteristic amount representing the size of thecharacter may be obtained in addition to said fourth characteristicamount. By adding the third characteristic amount representing the sizeof S the character, the restoration process of a deteriorated sectiondue to missing pixels, which increases in proportion to the size of thecharacter, can be performed more effectively. Thus, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process.

Furthermore, in a third characteristic amount representing the size ofthe character may be obtained in addition to said fifth characteristicamount. By adding the third characteristic amount, more effectiveexecution of the image quality improvement process is enabled in asimilar manner as above.

In addition, a third characteristic amount representing the size of thecharacter may be obtained in addition to said fourth and fifthcharacteristic amount. In a similar manner as above, addition of thethird characteristic amount enables more effective execution of theimage quality improvement process. Moreover, by considering the thirdcharacteristic amount after determining the image quality using twotypes of characteristic amounts, the fourth and the fifth characteristicamounts, a more effective execution of the image quality improvementprocess is enabled.

Moreover, if different regions exist on the original to be processed,restrictions may be imposed on the ranges where said characteristicamount is extracted, and the evaluation value may be computed byextracting a characteristic amount from each range. Thus, computation ofan evaluation value matching each range is enabled, even when differenttypes of range, such as gothic fonts and mincho fonts, exist next toeach other. Since an evaluation value matching each range is necessaryin performing the image quality improvement process and the binarycoding process, computation of an evaluation value matching each rangebecomes very effective.

Furthermore, by providing an image quality computation unit to extract acharacteristic amount to determine the image quality of the image dataentered by an image input device and to compute an evaluation valuewhich is the extracted characteristic amount; and an image qualityimprovement process to determine, from the characteristic amounts,candidates for the image quality improvement process by extractingsections which have the possibility of deteriorated image quality, andto execute the image quality improvement process on candidates for imagequality improvement processing by using evaluation values obtained bysaid image quality computation unit, accurate determination ofdeteriorated image sections is enabled. Moreover, missing pixels in thedeteriorated section are determined and the interpolation process ofpixels is enabled only when pixels are missing due to deterioration;thus, accurate restoration of deteriorated pixels is realized.Therefore, by applying this to a copy machine, when an original hasfaintness, for example, the present invention produces a higher qualitycopy than the original by restoring faint sections in the original;though, in general, the copied original has the same or worse faintnessthan the original. Moreover, while the present invention executesprocesses while determining image quality, it has no effect on anoriginal with a high quality image, leaving the high quality original asit is. Also, the present invention can be applied to a characterrecognition device to obtain a high rate of recognition.

In addition, a processing candidate extraction device to extractcandidates for image quality improvement processing, and pixelprocessing to interpolate pixels in executing image quality improvementon the processing candidates extracted by the processing candidateextraction device may be provided, and only sections with deterioratedimage quality are extracted, and the image quality improvement processmay be performed using the evaluation values computed by said imagequality computation unit. Thus, the image quality improvement process isperformed only when the image quality is poor, preventing, erroneousoperations such as addition of new pixels in a section which wasinitially blank.

Moreover, by providing a characteristic point extraction device todetect and extract the characteristic points generated by deteriorationin a section with deteriorated image quality, and a candidatedetermination device to determine candidates for image qualityimprovement using the positional relationship of the characteristicpoints extracted by the characteristic point extraction device,characteristic points from pixels facing each other across an emptyspace produced by deterioration may be extracted and, using thepositional relationship of the extracted characteristic points,determination of candidates for the image improvement process isenabled. Hence, deteriorated sections where the image improvementprocess is performed are extracted effectively and accurately.

Furthermore, a threshold computation means may be provided which, usingthe evaluation value obtained by said image quality computation unit,obtains a threshold value from a function with the evaluation value as avariable, compares said threshold value with the interval on whichinterpolation of pixels is performed, and determines whether or not toperform the interpolation process of pixels based on the results of thecomparison. Hence, the interpolation process may be performed only ondeteriorated sections due to missing pixels according to the degree ofimage quality deterioration, preventing, erroneous operations such asaddition of new pixels in a section which was initially blank.

Furthermore, a character cutting-out means may be provided and pixelinterpolation may be performed within the region of characters which arecut out by the character cutting-out means to improve the image quality.Hence erroneous pixel interpolation operation between adjacentcharacters may be prevented.

In addition, an image quality computation unit to extract acharacteristic amount to determine the image quality of the image dataentered by an image input device and to compute the characteristicamount as an evaluation value, and a binary threshold determinationmeans to determine the binary threshold value for the image to beprocessed using the evaluation value obtained by the image qualitycomputation unit may be provided. Hence, a binary threshold valuesuitable for the character is determined, minimizing faintness andsmudging of the image after the binary coding process.

Moreover, the binary threshold value determination process may define athreshold value which, in case the evaluation value is obtained byadding the first evaluation value and the second evaluation value, amongthe evaluation values obtained by said image quality computation unit,corresponds to the section making the evaluation value obtained byadding the two evaluation values minimum as the target binary thresholdvalue. Hence, the binary threshold value producing the best imagequality may be determined easily, and the binary threshold valuedetermination process may be simplified.

Furthermore, an image quality computation unit to extract acharacteristic amount to determine the image quality of the image dataentered by an image input device and to compute the characteristicamount as an evaluation value, a binary threshold determination means todetermine a binary threshold value for the image to be processed usingan evaluation value obtained by the image quality computation unit, andan image quality improvement unit to determine candidates for the imagequality improvement unit by extracting sections which have thepossibility of image quality deterioration based on the characteristicsand to perform the image quality improvement unit on candidates forimage quality improvement processing using the evaluation value obtainedby said image quality computation unit may be provided, enablingdetermination of a binary threshold value suitable to the character.Hence, faintness and smudging of images after the binary coding processmay be minimized and the deteriorated section may be further improved.Application of the present invention to copy machines, characterrecognition devices, and the like will produce good results.

While this invention has been described in conjunction with specificembodiments thereof, it is evident that many alternatives, modificationsand variations will be apparent to those skilled in the art.Accordingly, preferred embodiments of the invention as set forth hereinare intended to be illustrative, not limiting. Various changes may bemade without departing from the spirit and scope of the invention asdefined in the following claims.

What is claimed is:
 1. An image processing method for an imageprocessing device, comprising:inputting binary coded pixel characterimage data to the image processing device; dividing the binary codedpixel character image data into a plurality of data blocksrepresentative of blocks of pixels of an image; performing a firstorthogonal transformation with respect to the data blocks; determiningan evaluation value based on the first orthogonal transformationperformed with respect to the data blocks; modifying pixel values of thebinary coded pixel character image data based on the evaluation value toproduce modified binary coded pixel character image data; and outputtingthe modified binary coded pixel character image data.
 2. A methodaccording to claim 1, further including performing a second orthogonaltransformation with respect to said data blocks, wherein said evaluationvalue is a function of one of:a sum of the first and second orthogonaltransformations; and a difference between the first and secondorthogonal transformations.
 3. A method according to claim 1, whereinmodifying the pixel values of the binary coded pixel character imagedata comprises correcting the pixel values of the binary coded pixelcharacter image data based on the evaluation value to produce correctedbinary coded pixel character image data that more closely corresponds toan ideal version of the image.
 4. A method according to claim 1, whereinthe modified binary coded pixel character image data is corrected binarycoded pixel character image data that more closely corresponds to anideal version of the image.
 5. A method according to claim 1, whereinmodifying the pixel values of the binary coded pixel character imagedata comprises improving the pixel values of the binary coded pixelcharacter image data based on the evaluation value to produce improvedbinary coded pixel character image data that more closely corresponds toan ideal version of the image.
 6. A method according to claim 1, whereinthe modified binary coded pixel character image data is improved binarycoded pixel character image data that more closely corresponds to anideal version of the image.
 7. An image processing device comprising:animage input device that inputs binary coded pixel character image datato the image processing device; an image quality computation unit thatdivides a plurality of pixels of the binary coded pixel character imagedata into blocks of pixels, determines an evaluation value based on anorthogonal transformation performed with respect to the blocks ofpixels, and determines input image quality based on the evaluationvalue; an image quality improvement unit that determines a firstplurality of sections of the binary coded pixel character image data aspotential deteriorated sections and modifies pixel values of the binarycoded pixel character image data in at least some of said firstplurality of sections to produce modified binary coded pixel characterimage data based on said evaluation value; and an image output devicethat outputs said modified binary coded pixel character image data. 8.An image processing device according to claim 7, wherein said evaluationvalue is a function of one of a sum and a difference of orthogonaltransformations performed with respect to the blocks of pixels.
 9. Animage processing device according to claim 7, wherein the image qualityimprovement unit corrects the pixel values of the binary coded pixelcharacter image data in at least some of said first plurality ofsections based on said evaluation value to produce corrected binarycoded pixel character image data that more closely corresponds to anideal version of the image.
 10. An image processing device according toclaim 7, wherein the modified binary coded pixel character image data iscorrected binary coded pixel character image data that more closelycorresponds to an ideal version of the image.
 11. An image processingdevice according to claim 7, wherein the image quality improvement unitimproves the pixel values of the binary coded pixel character image datain at least some of said first plurality of sections based on saidevaluation value to produce improved binary coded pixel character imagedata that more closely corresponds to an ideal version of the image. 12.An image processing device according to claim 7, wherein the modifiedbinary coded pixel character image data is improved binary coded pixelcharacter image data that more closely corresponds to an ideal versionof the image.
 13. An image processing method for an image processingdevice, comprising:inputting binary coded pixel character image data tothe image processing device; dividing the binary coded pixel characterimage data into a plurality of data blocks representative of blocks ofpixels of an image; performing a first orthogonal transformation withrespect to the data blocks; determining an evaluation value based on thefirst orthogonal transformation performed with respect to the datablocks; correcting the binary coded pixel character image data based onthe evaluation value to produce corrected binary coded pixel characterimage data that more closely corresponds to an ideal version of theimage; and outputting the corrected binary coded pixel character imagedata.
 14. An image processing device comprising:an image input devicethat inputs binary coded pixel character image data to the imageprocessing device; an image quality computation unit that divides aplurality of pixels of the binary coded pixel character image data intoblocks of pixels, determines an evaluation value based on an orthogonaltransformation performed with respect to the blocks of pixels, anddetermines input image quality based on the evaluation value; an imagequality improvement unit that determines a first plurality of sectionsof the binary coded pixel character image data as potential deterioratedsections and corrects the binary coded pixel character image data in atleast some of said first plurality of sections to produce, based on saidevaluation value, corrected binary coded pixel character image data thatmore closely corresponds to an ideal version of the image; and an imageoutput device that outputs said corrected binary coded pixel characterimage data.
 15. An image processing method for an image processingdevice, comprising:inputting binary coded pixel character image data tothe image processing device; dividing the binary coded pixel characterimage data into a plurality of data blocks representative of blocks ofpixels of an image; performing a first orthogonal transformation withrespect to the data blocks; determining an evaluation value based on thefirst orthogonal transformation performed with respect to the datablocks; improving the binary coded pixel character image data based onthe evaluation value to produce improved binary coded pixel characterimage data that more closely corresponds to an ideal version of theimage; and outputting the corrected binary coded pixel character imagedata.
 16. An image processing device comprising:an image input devicethat inputs binary coded pixel character image data to the imageprocessing device; an image quality computation unit that divides aplurality of pixels of the binary coded pixel character image data intoblocks of pixels, determines an evaluation value based on an orthogonaltransformation performed with respect to the blocks of pixels, anddetermines input image quality based on the evaluation value; an imagequality improvement unit that determines a first plurality of sectionsof the binary coded pixel character image data as potential deterioratedsections and improves the binary coded pixel character image data in atleast some of said first plurality of sections to produce, based on saidevaluation value, improved binary coded pixel character image data thatmore closely corresponds to an ideal version of the image; and an imageoutput device that outputs said corrected binary coded pixel characterimage data.